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Abstract 

Satellite  distance  education  delivery  typically  has  one  main  drawback  —  limited 
student-to-student  and  student-to-instructor  interaction.  Telecommunications 
technology,  more  specifically  computer-mediated  communication  (CMC),  has  made  it 
possible  for  educational  institutions  to  overcome  this  drawback.  This  research  examines 
the  comparative  effectiveness  of  CMC  supplemented  satellite  distance  education  delivery 
and  traditional  face-to-face  education  delivery.  The  three  dependent  variables  that  were 
measured  to  compare  the  two  educational  delivery  methods  included:  performance, 
interaction,  and  attitude  outcomes.  Pre-  and  post-course  instruments  were  administered  in 
a  non-equivalent  quasi-experimental  design  to  students  enrolled  in  an  Air  Force  Institute 
of  Technology’s  School  of  Systems  and  Logistics  software  engineering  Professional 
Continuing  Education  course.  Descriptive  statistics  and  parametric  tests  were  used  to 
analyze  these  results  (at  alpha  =  .05).  The  parametric  tests  indicated  that  the  distance 
education  group  and  traditional  group  differed  significantly  on  two  demographic  areas, 
educational  level  and  self-rated  computer  proficiency,  and  on  the  pretest,  final  exam  and 
overall  course  grades.  The  distance  education  and  traditional  group  were  not  significantly 
different  on  pre-course  attitudes  towards  computers,  midterm  exam  grades,  and  post¬ 
course  instructor  and  course  ratings.  It  was  also  discovered  that  distance  education 
students’  mailing  list  participation  rates  did  not  differ  between  the  distance  education 
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students  with  more  positive  pre-course  attitudes  and  student  with  less  positive  pre-course 
attitudes. 

Despite  some  negative  findings  in  relation  to  course  performance,  the  overall  results 
of  this  study  were  encouraging.  They  suggest  that  satellite  distance  education  that  is 
supplemented  with  computer-mediated  communication  can  be  an  effective  educational 
delivery  method  for  software  engineering  Professional  Continuing  Education  courses. 
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THE  COMPARATIVE  EFFECTIVENESS  OF  COMPUTER-MEDIATED 


COMMUNICATION  SUPPLEMENTED  SATELLITE  DISTANCE  EDUCATION  AND 
TRADITIONAL  FACE-TO-FACE  EDUCATION 


1.  Introduction 


Introduction 

Telecommunications  technology  has  made  satellite  distance  education  more  feasible 
and  more  economical  for  corporations  and  universities  to  deliver  training  and  education  to 
non-local  students.  With  enormous  growth  in  this  area,  it  is  important  to  research  the 
effectiveness  of  distance  education  because  traditional  face-to-face  education  continues  to 
be  considered  by  many  as  the  ‘“industry  standard’  for  good  quality  education”  (Phelps 
and  others,  1991 : 14).  This  thesis  compares  the  effectiveness  of  an  Air  Force  Professional 
Continuing  Education  (PCE)  satellite  distance  education  course  that  is  supplemented  with 
computer-mediated  communication  (CMC)  to  the  traditional  face-to-face  course  delivery 
method. 

The  literature  dealing  with  satellite  distance  education  in  various  adult  populations 
(undergraduate  students,  graduate  students,  non-credit  students,  and  military  personnel), 
consistently  indicates  that  the  delivery  of  educational  programming  via  satellite  broadcast 
is  “educationally  effective”  (Moore,  1990:11).  However,  standard  satellite  distance 
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education  methods  have  always  had  one  main  drawback  —  limited  student-to-student  and 
student-to-instructor  interaction  (Jegede  and  others,  1994:91).  Some  academic 
institutions  are  taking  advantage  of  computer-mediated  communications  to  help  overcome 
the  problem  of  limited  interactivity.  Computer-mediated  communication  occurs  when  two 
or  more  people  communicate  electronically  over  a  computer  network.  The  interaction 
potential  of  computer-mediated  communication  appears  to  be  a  beneficial  supplement  to 
satellite  distance  education. 

Background 

The  Air  Force  Institute  of  Technology’s  School  of  Systems  and  Logistics  Software 
Engineering  Department  (AFIT/LSS)  at  Wright-Patterson  AFB,  Ohio,  recently  developed 
a  computer-mediated  communication  portion  to  supplement  their  satellite  distance 
education  courses.  This  is  the  first  deliberate  attempt  by  the  Air  Force  Institute  of 
Technology  (AFIT)  to  electronically  communicate  with  its  distance  education  students. 
Many  research  efforts  have  examined  the  effectiveness  of  satellite  distance  educational 
delivery  and  the  effectiveness  of  computer-mediated  communication  educational  delivery. 
However,  none  have  examined  the  effectiveness  of  the  satellite  broadcast  used 
simultaneously  with  computer-mediated  communication  in  the  Air  Force  Professional 
Continuing  Education  (PCE)  arena.  In  the  many  resources  consulted,  studies  were  also 
abundant  in  the  areas  of  distance  education  at  civilian  institutions  using  CMC.  However, 
studies  of  CMC  distance  education  in  the  Air  Force  PCE  arena  were  also  lacking.  Since 
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this  combination  of  distance  education  delivery  is  new  to  the  Air  Force,  it  is  deserving  of 
research  attention. 

Some  noteworthy  distance  education  efforts  in  the  military  include  the  Air  Force  Air 
Command  and  Staff  College,  the  Army  Reserve  Component,  and  the  Department  of 
Defense  Dependent  schools.  The  Air  Force  recently  developed  an  online  Professional 
Military  Education  course  for  majors  and  major  selectees  (Air  Command  and  Staff 
College),  but  the  results  of  their  efforts  have  not  been  published  at  this  time  (Taylor, 
1995:2).  The  US  Army  Reserve  Component  conducted  a  study  on  the  effectiveness  and 
costs  of  distance  education  using  computer-mediated  communication  in  1990.  They  found 
that  students  taking  the  CMC  course  “achieved  test  scores  no  different  or  better  than 
those  of  their  resident  counterparts”  (Phelps  and  others,  1991: 14).  The  Department  of 
Defense  Dependent  Schools  (DoDDS)  have  developed  distance  education  courses  using 
computer  conferencing,  video  taped  instruction,  and  computer  tutorials.  DoDDS  serves  a 
student  population  of  over  150,000,  making  it  the  largest  American  school  district.  Most 
of  DoDDS  schools  are  relatively  small  (having  between  400  to  700  students  each)  so 
offering  a  variety  of  classes  is  sometimes  impractical.  Distance  education  was  seen  as  a 
tool  to  bring  specialty  courses,  such  as  Pascal  computer  programming  and  calculus,  to  the 
DoDDS  students  who  are  scattered  over  eleven  different  time  zones  (Morgan  and  Sheets, 
1992:60-61). 

Many  university  instructors  have  the  capability  to  reach  students  via  the  Internet, 
and  those  individuals  who  feel  comfortable  with  this  enviromnent  are  taking  advantage  of 
this  technology.  Richard  Smith,  co-author  of  Navigating  the  Internet,  firmly  believes  that 
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the  widespread  access  to  the  Internet  will  alter  the  way  we  educate.  He  states  that  "with 
the  collaboration  from  educators  from  interdisciplinary  backgrounds  and  from  diverse 
institutions  and  cultures,  education  will  change  from  the  traditional  environment  to  a 
virtual  classroom  with  no  walls"  (Smith,  1994:4).  Distance  education  technologies  seem 
to  be  expanding  at  such  a  rapid  pace  that  it  is  crucial  for  educators  to  continue  to  research 
the  comparative  effectiveness  of  distance  education  and  traditional  education  methods. 
With  the  new  distance  education  initiative  at  AFIT,  there  is  a  need  to  research  the 
effectiveness  of  CMC  supplemented  satellite  distance  education  in  comparison  to 
traditional  educational  delivery  in  the  Air  Force  PCE  arena. 

Problem  Statement 

The  Air  Force  Institute  of  Technology’s  School  of  Systems  and  Logistics  Software 
Engineering  Department  (AFIT/LSS)  has  developed  satellite  courses  for  four  of  its  five 
Software  Professional  Development  Program  (SPDP)  resident  courses.  A  traditional 
AFIT/LSS  Professional  Continuing  Education  course  is  offered  in  residence  for  two 
weeks  —  eight  hours  each  day.  The  five  courses  are  usually  taken  in  a  predetermined 
sequence.  The  first  course  in  the  series,  CSE  492,  is  a  prerequisite  for  all  of  the  remaining 
SPDP  courses  (CSE  493,  CSE  494,  CSE  495,  CSE  496).  Each  software  engineering 
course  is  graded  on  a  pass/fail  basis.  Students  are  required  to  achieve  an  overall  course 
grade  of  70%  to  pass.  The  AFIT/LSS  software  engineering  course  listing  can  be  found  at 
Appendix  A. 
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AFIT/LSS  has  adapted  the  majority  of  its  courses  to  be  offered  by  satellite  over  a  10 
week  quarter.  Each  class  meets  three  times  a  week  for  one  and  a  half  hours  a  day.  The 
typical  satellite  course  consists  of  a  live,  one-way  video  satellite  broadcast  from  the  AFIT 
host  site,  located  at  Wright-Patterson  AFB,  to  multiple  remote  sites.  Each  remote  site  is 
equipped  with  push-to-talk  microphones  that  allow  a  two-way  audio  connection  with  the 
host  site.  The  satellite  classroom  is  also  composed  of  traditional  face-to-face  students  at 
the  AFIT  host  site. 

Students  in  a  traditional  classroom  setting  are  able  to  take  advantage  of  interacting 
with  other  students  during,  before,  or  after  the  normal  class  session.  Traditional  students 
can  meet  with  fellow  classmates  and  the  instructor  in  face-to-face  meetings  to  clarify  and 
discuss  issues  brought  up  in  class.  Prior  to  this  initiative,  AFIT  distance  education 
students  had  little  or  no  interaction  among  themselves  outside  the  classroom  setting. 

Since  distance  education  students  may  not  have  the  opportunity  to  hold  face-to-face 
discussion  groups  with  other  students  and  cannot  meet  physically  with  the  instructor,  the 
Software  Engineering  Department  has  incorporated  computer-mediated  communication, 
in  the  form  of  an  electronic  mailing  list,  to  supplement  its  satellite  courses.  By  using  a 
mailing  list  (an  electronic  mailing  program  that  sends  all  posted  messages  to  each 
individual  who  is  subscribed  to  the  mailing  list)  to  provide  asynchronous  communication 
during  non-class  hours,  AFIT/LSS  has  tried  to  bridge  the  perceived  interaction 
deficiencies  of  typical  satellite  distance  education  courses. 

With  the  addition  of  this  online  educational  technology,  there  is  an  important  need  to 
evaluate  distance  education  courses  that  are  supplemented  with  computer-mediated 


5 


communication.  Is  satellite  distance  education  delivery  supplemented  with  CMC  as  good 
as  or  better  than  traditional  face-to-face  course  delivery?  With  the  demand  to  reach  more 
students,  it  is  imperative  to  find  a  suitable  distance  education  delivery  for  professional 
continuing  education  courses  that  not  only  allows  for  interaction  between  the  student  and 
the  instructor  during  classtime,  but  also  allows  for  the  student  to  conununicate  with  the 
instructor  and  other  classmates  during  non-class  periods. 

Research  Objective 

The  objective  of  this  research  was  to  examine  an  Air  Force  sofi:ware  engineering 
distance  education  course  to  determine  if  CMC  supplemented  satellite  distance  education 
delivery  is  as  good  as  or  better  than  the  traditional  face-to-face  education  delivery  method. 
In  particular,  were  the  CMC  supplemented  satellite  distance  education  course 
performance,  interaction,  and  attitude  outcomes  at  least  as  good  as  the  traditional  face-to- 
face  education  course  outcomes? 


Hypotheses 

This  research  effort  examined  the  performance,  interaction,  and  attitude  outcomes 
of  the  distance  education  group  and  traditional  group  by  testing  five  hypotheses.  The  first 
hypothesis  tested  for  significant  differences  on  performance  scores  between  the  two 
groups.  The  second  and  third  hypotheses  examined  the  two  groups  for  differences  on  the 
student-to-student  interaction  and  student-to-instructor  interaction  that  occurred  during 
non-class  hours.  Hypothesis  four  tested  whether  more  positive  pre-course  attitudes  led  to 
higher  mailing  list  participation  rates  for  the  distance  education  group.  The  last  hypothesis 
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(hypothesis  five)  tested  for  significant  differences  between  the  two  groups  on  course  and 
instructor  end-of-course  ratings.  The  five  research  hypotheses  are  as  follows: 


Hypothesis  1.  Ho:  There  will  be  no  significant  difference  between  the 
distance  education  and  traditional  students  on  the  scores  measuring  student 
performance. 

Ha:  There  will  be  a  significant  difference  between  the  distance  education 
and  traditional  students  on  the  scores  measuring  student  performance. 

Hypothesis  2.  Hq:  The  distance  education  students  will  not  report  more 
interaction  with  fellow  students  during  non-class  hours  than  the  traditional 
students. 

Ha:  The  distance  education  students  will  report  more  interaction  with 
fellow  students  during  non-class  hours  than  the  traditional  students. 

Hypothesis  3.  Ho:  The  distance  education  students  will  not  report  more 
interaction  with  the  instructor  during  non-class  hours  than  the  traditional 
students. 

Ha:  The  distance  education  students  will  report  more  interaction  with  the 
instructor  during  non-class  hours  than  the  traditional  students. 

Hypothesis  4a:  Ho:  Distance  education  students  with  more  positive  pre¬ 
course  attitudes  towards  computers  will  not  be  more  likely  to  have  higher 
mailing  list  participation  rates. 

Ha:  Distance  education  students  with  more  positive  pre-course  attitudes 
towards  computers  will  be  more  likely  to  have  higher  mailing  list 
participation  rates. 

Hypothesis  4b:  Ho:  Distance  education  students  with  more  positive  pre¬ 
course  expectations  towards  the  mailing  list  will  not  be  more  likely  to  have 
higher  mailing  list  participation  rates. 

Ha:  Distance  education  students  with  more  positive  pre-course 
expectations  towards  the  mailing  list  will  be  more  likely  to  have  higher 
mailing  list  participation  rates. 

Hypothesis  5a:  Ho;  There  will  be  no  significant  difference  between  distance 
education  and  traditional  students  on  the  post-course  survey  instructor 
ratings. 

Ha:  There  will  be  a  significant  difference  between  distance  education  and 
traditional  students  on  the  post-course  survey  instructor  ratings. 
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Hypothesis  5b:  Ho:  There  will  be  no  significant  difference  between 
distance  education  and  traditional  students  on  the  post-course  survey 
course  ratings. 

Ha!  There  will  be  a  significant  difference  between  distance  education  and 
traditional  students  on  the  post-course  survey  instructor  ratings. 

Methodology 

A  one-way  (one  factor-educational  mode)  non-equivalent  quasi-experimental  design 
was  used  for  this  research  effort.  The  experiment  design  and  statistical  analysis  were 
based  on  similar  efforts  by  Cheng  and  others,  as  detailed  in  their  article  titled, 

“Comparison  of  Performance  and  Attitude  in  Traditional  and  Computer  Conferencing 
Classes”  (Cheng  and  others,  1991:55). 

The  distance  education  group  received  the  satellite  broadcast  of  the  course  and 
participated  in  computer-mediated  communication.  The  traditional  group  received  the 
face-to-face  instruction  that  was  broadcast  to  the  distance  education  students.  The 
traditional  group  did  not  participate  in  the  CMC.  Both  the  traditional  group  and  distance 
education  group  had  access  to  the  taped  class  sessions,  in  the  event  that  they  were  unable 
to  attend  class  during  the  scheduled  time. 

Research  Instruments 

In  order  to  evaluate  the  research  questions,  data  was  collected  fi'om  each  student 
using  pre-and  post-course  survey  instruments.  Grade  data  was  provided  on  each  student 
by  the  instructor  in  the  following  areas:  pretest,  midterm  exam,  final  exam  and  overall 
course  grade.  The  pre-course  instruments  (the  pretest  and  pre-course  survey)  established  a 
baseline  for  assessing  learning  and  attitude  changes.  The  post-course  survey  provided 
interaction  and  attitudinal  data.  Questions  on  the  pre-  and  post-course  survey  were  used 
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with  permission  from  Starr  Roxanne  Hiltz’s  research  efforts  detailed  in  Online  Education: 
Perspectives  in  a  New  Environment  (Hiltz,  1990:134).  Participation  on  the  mailing  list 
was  monitored  and  was  used  to  determine  mailing  list  participation  rates  for  the  distance 
education  students. 


Research  Assumptions 

It  was  assumed  that  both  sets  of  students  were  subjected  to  the  same  educational 
material,  surveys,  tests  and  instructor  lessons.  Important  materials,  like  homework 
assignments,  that  were  given  to  the  distance  education  group  via  the  mailing  list  was 
presented  to  the  traditional  students  during  normal  classtime. 

Some  of  the  results  obtained  may  have  been  affected  by  the  technological 
experiences  encountered  with  the  satellite  broadcast  and  electronic  mailing  list.  If  a 
satellite  downlink  could  not  be  established,  not  only  do  those  remote  students  miss  class, 
and  have  to  get  the  class  on  video,  but  the  other  students'  classtime  was  also  interrupted 
while  the  instructor  dealt  with  the  problem. 

Because  this  course  was  a  software  engineering  class,  it  was  assumed  that  the 
performance,  interaction,  and  attitude  results  of  the  delivery  medium  were  influenced  by 
the  computer  literacy  of  the  student  population  and  the  nature  of  the  course. 

Scope  and  Limitations 

This  research  effort  concentrated  on  a  single  course  offering  of  a  Software 
Engineering  Professional  Continuing  Education  course.  Additional  research  in  this  area 
can  be  conducted  with  the  AFIT  Software  Engineering  PCE  courses  because  AFIT/LSS 
continues  to  supplement  its  satellite  distance  education  courses  with  an  electronic  mailing 
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list.  Results  from  this  experiment  can  be  applied  to  similar  courses  in  the  computer 
science/software  engineering  field. 

Management  Implications 

With  the  need  to  reach  more  students  with  less  money,  it  is  imperative  for  the 
military  to  develop  an  effective  distance  education  method.  Satellite  distance  education  is 
now  available  at  most  Air  Force  bases,  and  with  more  and  more  bases  becoming 
connected  to  the  Internet,  the  possibilities  for  interaction  using  computer-mediated 
communication  are  endless.  New  distance  education  initiatives  need  to  be  researched  and 
fine-tuned  in  order  for  the  military  to  receive  the  greatest  benefits  from  new  technology. 

Organization  of  the  Research 

This  thesis  is  organized  in  five  chapters.  This  chapter,  the  first  one,  covers  a  general 
introduction  of  the  research  experiment.  Chapter  two  defines  important  terminology  and 
reviews  the  literature  on  other  distance  education  research  projects.  The  third  chapter 
describes  the  methodology  used  for  this  experiment.  The  experiments'  results  and  data 
analysis  can  be  found  in  chapter  four.  The  fifth  and  final  chapter  details  the  conclusions 
and  recommendations  developed  by  the  researcher. 
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II.  Literature  Review 


Introduction 

This  chapter  provides  a  definition  of  distance  education,  presents  the  criteria  for 
evaluating  educational  delivery  and  discusses  the  characteristics  of  the  two  distance 
education  methods  (satellite  and  computer-mediated  communication  (CMC))  that  were 
compared  to  the  traditional  classroom  delivery  method. 

Definition  of  Distance  Education 

Distance  education  occurs  whenever  there  is  any  form  of  structured  education  in 
which  the  instructor  and  learner  are  separated,  either  physically  or  in  time,  and  one  or 
more  techniques  are  used  to  exchange  information  between  them  (Harris,  1991 : 19; 
Duning,  1993:273).  Distance  education  can  offer  many  of  the  benefits  of  traditional 
classroom  instruction:  routine  interaction  between  student  and  teacher,  regular  monitoring 
of  the  student's  progress,  and  varied  ways  of  presenting  course  material.  Many  different 
media  deliver  distance  education.  Some  examples  include:  correspondence,  radio, 
videotape,  open-broadcast  television,  cable  television,  satellite  broadcast,  computer 

networks,  electronic  blackboards,  video  conferencing,  and  teleconferencing.  The  most 

\ 

effective  distance  education  programs  use  a  combination  of  telecommunications  media 
that  suit  the  needs  of  their  individual  courses.  "The  proliferation  of  personal  computers 
and  the  growing  accessibility  of  the  Internet  and  other  computer  communication  services" 
have  made  computer-mediated  communication  more  feasible  for  both  students  and 
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teachers  (Barreau  and  others,  1994: 10).  The  main  difficulty  for  educators  is  not  attaining 
the  technological  resources,  but  selecting  the  best  and  most  economical  method  for  the 
delivery  of  their  particular  distance  education  program. 

Criteria  for  Evaluating  Educational  Delivery 

Comparison  of  two  or  more  media  in  relation  to  their  effectiveness  is  the  most 
common  type  of  distance  education  research.  With  hundreds  of  media  comparison  studies 
having  been  performed  over  the  last  forty  years,  the  results  have  been  fairly  solid:  the 
educational  method  does  not  “appear  to  make  any  important  difference  in  student 
achievement,  attitudes  and  retention”  (Willis,  1994:42).  Some  of  the  media  comparison 
studies  and  the  criteria  they  examined  are  detailed  in  the  following  paragraphs. 

The  most  basic  measurement  of  “desirable  outcomes  for  a  course  is  mastery  of  the 
fundamental  facts,  concepts,  and  skills  that  the  course  is  designed  to  teach”  (Hiltz, 

1990: 144).  Mastery,  or  achievement,  is  usually  tested  by  assignments  and  exams  that  are 
graded  by  the  professor.  However,  even  though  student  achievement  as  measured  by 
course  examinations  "is  an  important  measure  of  the  success  of  an  educational  program,  it 
is  almost  never  the  only  important  measure"  (Kinnaman,  1992:18).  Some  other  important 
and  usefol  evaluation  data  to  consider  include  performance  self-assessment,  student  and 
teacher  attitudes  toward  the  course,  and  student  and  teacher  behaviors  during  the  course. 
Performance  assessment  focuses  on  what  students  can  do  —  i.e.,  complete  a  project 
successfully.  Student  attitudes  and  behavior  that  are  measured  by  a  survey  or  observation 
can  give  insight  into  the  degree  a  program  affects  students'  effort  or  motivation.  Teacher 


12 


attitudes  and  behavior  towards  the  course  and  how  it  may  or  may  not  have  influenced  the 
students'  performance  or  attitude  is  another  important  factor  to  consider  when  evaluating 
the  effectiveness  of  a  course  (Kinnaman,  1992:18). 

Verduin  and  Clark  credit  D.  Cooler  for  the  basic  schemata  for  assessing  distance 
education  program  quality  and  effectiveness  in  their  book  Distance  Education  (Verduin 
and  Clark,  1991 :88).  The  assessment  criteria  includes:  1)  learner  outcomes  (cognitive 
skills,  psychomotor  skills,  affective  skills  and  dropout  rate);  2)  access;  3)  quality ;  4)  cost 
effectiveness  and  efficiency;  5)  impact;  6)  relevance  to  needs  and  expectations;  5) 
generation  of  knowledge;  and  6)  acceptability  (status  and  attitudes).  Verduin  and  Clark’s 
extensive  review  of  the  current  research  revealed  distance  education  achieves  most  learner 
outcomes  (cognitive,  affective  and  psychomotor)  equal  to  those  achieved  by  traditional 
education.  The  dropout  rate  for  distance  education  is  higher  than  for  traditional  education 
and  many  institutions  are  making  efforts  to  provide  better  learning  materials  and  support 
systems  to  combat  this  problem. 

Important  factors  in  testing  the  outcomes  of  distance  education  projects,  as  detailed 
by  Hudson  and  Boyd,  include  student  performance,  performance  of  hardware  and 
software;  complexity  of  materials'  development;  and  attitudes  of  teachers,  learners,  and 
project  staff  (Hudson  and  Boyd,  1984:12).  The  performance  of  hardware  and  software  is 
a  factor  that  needs  to  be  considered  when  evaluating  student  attitudes.  Frustration  with 
the  course  media  may  be  attributed  to  sites  not  receiving  the  satellite  broadcast  or  a 
student  not  having  convenient  access  to  a  suitable  computer  for  computer-mediated 
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communication.  Instructors  also  need  to  keep  the  delivery  mode  and  audience  in  mind 
when  developing  their  course  materials. 

In  the  book  Contemporary  Issues  in  American  Distance  Education.  Peter  Cookson 
of  Penn  State  University  defines  learners'  success  as  one  or  more  of  the  following 
outcomes: 

1.  Persistence; 

(a)  provisional  registration  followed  by  final  registration, 

(b)  completion  of  all  course  assignments  prior  to  final  examination, 

(c)  earning  of  credit  following  the  passing  of  the  final  examination. 

2.  Academic  achievement; 

(a)  percentage  of  course  assignments  successfully  competed, 

(b)  final  course  grade. 

3.  Satisfaction  with  the  course  learning  experience. 

4.  Intention  to  enroll  in  additional  distance  education  courses. 

By  keeping  in  mind  these  learners'  success  outcomes,  Mr.  Cookson  believes  that  a 
successful  distance  education  course  can  be  designed  to  attain  certain  specified  and 
desired  outcomes  (Cookson,  1990:192). 

Each  distance  education  medium  incorporates  different  technologies  and  thus,  the 
distance  education  methods  that  are  being  researched  are  deserving  of  individual  attention 
to  the  specific  characteristics  of  each  medium. 


Characteristics  of  Satellite  Distance  Education 

Satellite  networks  became  possible  in  the  early  1980s,  when  Hewlett  Packard  built  a 
satellite  network  to  link  California  with  over  one  hundred  downlink  sites  in  the  U.S.  and 
Canada.  Most  satellite  networks  consist  of  one-way  video  from  the  transmitting  site  to 
the  receiving  site,  with  a  two-way  audio  connection  being  handled  by  push-to-talk 
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microphones,  videophones,  or  telephones.  The  satellite  signal  can  be  transmitted 
terrestrially  via  microwaves,  telephone  lines,  local  cable  systems  or  broadcast  stations. 
Typically,  students  are  sent  printed  material  (either  through  the  mail  or  via  computer)  such 
as  the  syllabi,  assignments,  course  material  and  class  notes. 

According  to  Bruce  Barker  and  Marvin  Flatten,  in  their  article  "Student 
Perceptions  on  the  Effectiveness  of  College  Credit  courses  taught  via  Satellite,"  the 
traditional  method  of  face-to-face  educational  delivery  must  be  modified  to  take  into 
account  the  satellite's  interaction  parameters.  An  instructor  can  teach  to  a  television 
camera  either  with  or  without  students  in  his  or  her  satellite  classroom.  Using  one-way 
video  and  two-way  audio,  the  instructor  is  seen  and  heard  by  the  remote  students,  but  the 
instructor  cannot  see  the  remote  students.  Students  in  the  remote  classrooms  will  only  be 
heard  by  the  instructor  if  those  students  initiate  contact  with  the  instruction  site.  Once 
communication  has  been  initiated  with  the  host  site,  the  students  at  the  other  remote  sites 
and  the  host  site  can  hear  that  particular  student’s  comment  (Barker,  1988:45). 

One  of  the  major  advantages  of  satellite  courses  is  the  ability  to  reach  large 
audiences.  Reaching  a  large  number  of  students  at  a  given  time  ensures  standardization  of 
course  material  among  those  students  (Kline,  1993:50).  Some  of  the  disadvantages 
include:  remote  students  are  not  seen  by  the  instructor  and  often  feel  alienated  or  not 
included  in  the  class  discussions  and  are  disadvantaged  by  the  "lack  of  opportunity  of 
dialogue,  debate,  conversational  learning,  and  collaborative  work"  (Odasz,  1992:70). 
Because  the  instructor  cannot  evaluate  remote  students  based  on  visual  evaluation,  some 
programs  have  used  a  facilitator  to  fill  that  role. 
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In  a  study  that  examined  a  satellite  broadcast  course  that  was  supplemented  with 
computer-mediated  communication,  Deborah  Barreau  and  her  colleagues  worked  on  the 
Group  Collaboration  in  the  Virtual  Classroom  effort.  They  tested  the  hypothesis  that 
"remote  [distance  education]  students  will  not  differ  from  on-campus  students  in  their 
reporting  of  successful  collaboration  in  the  virtual  classroom."  They  reported  that  the 
majority  of  the  remote  and  on-campus  students  reported  high  levels  of  satisfaction  in  the 
virtual  classroom.  However,  some  problems  in  sharing  information  and  the  feeling  of 
isolation  were  reported  by  some  remote  students  who  were  not  familiar  with  CMC 
(Barreau  and  others,  1994:10). 

Barry  Willis,  author  of  Distance  Education:  Strategies  and  Tools,  points  out  the 
reality  of  satellite  distance  education  —  that  “students  interact  infrequently  with  instructors 
both  on  and  off  the  air”  (Willis,  1994:47).  With  the  advent  of  telecommunications 
technologies  that  allow  more  access  to  student-to-student  and  student-to-instructor 
communication,  studying  distance  education  projects  that  use  computer-mediated 
communication  is  very  important.  Some  studies  have  found  asynchronous  interaction 
from  computer-mediated  communication  both  “very  useful  and  effective”  (Willis 
1994:49).  The  characteristics  of  CMC  are  detailed  in  the  next  section. 

Characteristics  of  Computer-Mediated  Distance  Education 

Computer-mediated  distance  education  occurs  over  some  type  of  computer 
network  —  either  local  or  global  (the  Internet).  Most  computer  networks  consist  of 
microcomputers,  modems  (a  device  that  converts  digital  computer  signal  into  analog 
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format  for  transmission  over  the  telephone  lines)  and  a  host  computer.  Together,  they 
transmit  and  receive  information  over  telephone  wires.  The  host  computer  acts  as  a 
clearinghouse  -  receiving,  storing,  sorting  and  presenting  the  material  to  the  appropriate 
users.  Usually,  the  types  of  communications  media  used  in  computer  networks  are 
electronic  mail  (e-mail),  electronic  mailing  lists,  bulletin  board  systems  (bbs),  and 
computer  conferencing  (Selfe,  1994:1 16).  Electronic  mail  is  “characterized  by  storage  of 
a  message  at  an  ‘electronic  address’  that  can  be  received  by  the  recipient  via  a 
telecommunication-equipped  personal  computer”  (Willis,  1994:295).  This  networking 
application  allows  users  to  send  and  receive  mail  messages  electronically.  An  electronic 
mailing  list  is  a  type  of  group  e-mail  that  subscribes  individuals  that  have  an  e-mail  address 
and  permits  sending  and  receiving  e-mail  messages  to  and  from  everyone  subscribed  to  the 
mailing  list.  A  mailing  list  server  computer  receives  e-mail  from  the  subscribers  that  is 
intended  for  posting.  The  server  then  duplicates  the  message  and  delivers  it  to  everyone 
subscribed  to  that  mailing  list.  Electronic  mailing  list  programs  can  be  either 
unmoderated,  where  the  mailing  list  program  does  everything  automatically,  or 
moderated,  where  an  individual  manually  approves  all  of  the  requests  for  subscriptions  and 
postings  using  the  mailing  list  program  commands.  Bulletin  board  systems  are  electronic 
storage  areas  arranged  by  topics,  where  remote  users  can  read  and  post  electronic 
messages  to  the  bbs  of  their  choice.  Computer  conferencing  enables  individuals  at 
different  locations  the  ability  to  communicate  directly  with  each  other  in  a  “shared 
electronic  space”  (Selfe,  1994:116). 
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Each  online  medium  links  the  instructor  with  the  student  located  at  different  sites  in 
a  different  manner  (Gransden,  1994:A41).  Some  are  asynchronous,  two-way 
communication  in  which  there  is  a  time  delay  between  when  a  message  is  sent  and  when  it 
is  received.  Some  examples  are  e-mail,  mailing  lists  and  bbs.  Computer  conferencing  is 
synchronous  or  real-time,  because  all  participants  are  online  at  the  same  time. 

Computer-mediated  distance  education  removes  barriers  such  as  social  and  cultural 
diversity  that  are  common  in  the  traditional  classroom  educational  setting  (Smith,  1:3). 
Some  educators  believe  that  distance  education  will  eventually  give  most  students 
practically  unlimited  course  options  from  universities  across  the  globe,  "regardless  of 
where  they  are  formally  enrolled"  (Jacobson,  1994:A23). 

With  an  opportunity  for  every  student  to  have  an  equal  chance  to  participate,  there  is 
a  democratic  element  inherent  in  many-to-many  computer-mediated  communication 
(Lewis  and  Hedegaard,  1993:69).  Status  and  role  differences  tend  to  be  less  of  a  factor  in 
online  environments.  It  is  also  very  difficult  for  students  to  hide  in  CMC  environments. 
“Social  loafing  becomes  more  conspicuous  because  an  uninvolved  group  member  literally 
disappears”  (Lewis  and  Hedegaard,  1993:69).  For  these  reasons,  CMC  interaction  tends 
to  be  more  evenly  distributed  when  compared  to  traditional  group  interaction. 

Some  characteristics  of  computer-mediated  communication  attained  in  a  study  at  the 
Houston  Community  College  System  (HCCS)  include:  (1)  Immediacy  -  especially 
compared  to  print-based  correspondence  distance  education;  (2)  Sense  of  group  identity  - 
-  the  computer  system  became  a  meeting  place  for  students;  (3)  Improved  dialogue  — 
students  interact  more  compared  to  the  traditional  classroom  instructional  setting;  (4) 
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Improved  instructor  control  —  the  computer  system  can  log  activities;  and  (5)  Active 
learning—  student  participation  improved  (Smith,  1994:3). 

In  The  Emerging  Worldwide  Electronic  University.  Parker  Rossman  states  that 
computer-mediated  instruction  can  be  as  effective  as  traditional  classroom  methods. 
Comparing  the  results  of  online  courses  versus  traditional  courses,  "it  was  found  that  the 
use  of  the  computer-empowered  technology  actually  improved  the  quality  of  education." 
In  a  computer-mediated  environment,  the  student  is  not  restricted  by  the  limited  time 
available  to  ask  questions  in  a  traditional  classroom  environment.  In  front  of  his  or  her 
computer,  the  student  can  insert  a  question,  or  speak,  as  often  as  he  or  she  desires. 
Online  communication  encourages  students  to  participate  more  and  thus,  allows  them  to 
be  more  in  control  of  their  environment  (Rossman,  1992:91).  The  students  were  more 
actively  involved  in  self-teaching  as  well  as  in  joint  study  projects  with  fellow  students. 
Online  students  also  report  that  they  kept  in  closer  touch  with  their  teachers,  as  much  as, 
or  more  than  those  students  who  attended  traditional  college  classes.  Mr.  Rossman  also 
discussed  Starr  Roxanne  Hiltz's  research  efforts  in  the  area  of  online  distance  education. 
Ms.  Hiltz  found  that  online  students  (Rossman,  1992:122-123): 

(1)  Have  test  scores  as  good  as  those  in  conventional  classrooms  if  the 
quality  of  instruction  is  the  same; 

(2)  Report  improved  access  to  instructors  and  convenient  access  to  need 
courses  and  educational  experiences  if  they  have  adequate  technology  to 
use;  and 

(3)  Tend  to  develop  improved  ability  to  collaborate  and  communicate  with 
other  students. 
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Some  studies  have  shown  computer  literate  individuals  will  be  more  accepting  of 
computer-mediated  communication  technology  and  their  overall  attitude  towards  the  class 
medium  will  be  better  on  average  than  those  who  are  not  as  computer  literate  (Hiltz, 
1993:89).  Research  has  also  indicated  that  distance  education  computer-mediated  courses 
tend  to  have  the  best  performance  outcomes  when  the  course  is  computer-related  (Hiltz, 
1993:94). 

S.  R.  Hiltz  also  cautions  researchers  not  to  equate  “lack  of  use”  of  the  medium  by 
the  student  with  rejection  of  the  medium  (Hiltz,  1989:387).  The  student  may  not  have 
“convenient  access”  or  sufficient  understanding  of  the  medium.  She  also  stresses  that 
requiring  students  to  use  the  medium  may  increase  usage  statistics,  while  at  the  same  time, 
building  student  resentment  and  dissatisfaction  toward  the  medium  (Hiltz,  1989:387). 

Ms.  Hiltz  warns  that  “future  studies  of  CMC  should  not  assume  that  usage  alone  or 
subject  satisfaction  alone  are  adequate  indicators  of  acceptance  or  success”  (Hiltz, 
1989:397). 

In  a  summary  provided  by  S.R.  Hiltz  of  twelve  previous  studies  of  computer- 
mediated  communication  acceptance  (in  terms  of  amount  of  use  of  the  system),  some  of 
the  variables  that  best  predicted  CMC  acceptance  include:  previous  experience, 
expectations  about  the  systems,  attitudes  towards  computers,  own  versus  shared  terminal, 
anticipated  usefulness,  and  available  night  or  weekend  hours  to  use  the  CMC  (Hiltz, 
1990:743). 
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Summary 

The  effectiveness  of  distance  education  delivery  continues  to  be  a  topic  of  debate. 
Many  studies  have  come  to  the  conclusion  that  distance  education  delivery  is  as  effective 
as  traditional  education  methods.  Since  there  are  so  many  institutions  turning  to  distance 
education  to  reach  a  larger  audience,  the  effectiveness  of  distance  education  methods 
merits  further  study  (Souder,  1993:37). 

This  chapter  defined  distance  education  and  gave  examples  of  distance  education 
media.  Criteria  for  evaluating  educational  delivery  and  the  characteristics  of  satellite  and 
computer-mediated  communication  distance  education  were  also  discussed.  The  focus  of 
this  research  effort  will  be  on  three  dependent  variables  --  performance,  interaction,  and 
attitude  outcomes.  Performance  and  attitude  outcomes  were  chosen  because  of  the 
number  of  media  comparison  studies  that  used  these  variables  to  determine  whether 
distance  education  is  as  good  as  or  better  than  traditional  education.  Interaction  outcomes 
were  chosen  because  of  the  new  initiative  to  supplement  the  satellite  broadcast  with  a 
mailing  list.  In  particular,  do  the  more  positive  distance  education  students  (in  terms  of 
expectations  of  the  CMC  system  and  pre-course  attitudes  towards  computers)  have  higher 
participation  rates?  These  variables  also  appear  to  be  the  most  measurable  in  the  pre¬ 
course/post-course  survey  and  test  format.  The  next  chapter  will  explain  the  methodology 
that  was  chosen  to  study  these  dependent  variables. 
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III.  Methodology 


Introduction 

This  chapter  will  discuss  the  non-equivalent  control  group  methodology  that  was 
used  to  analyze  the  performance,  interaction,  and  attitude  outcomes  of  the  Air  Force 
Institute  of  Technology’s  School  of  Systems  and  Logistics  Software  Engineering 
Departments’  (AFIT/LSS)  Object  Oriented  Analysis  and  Design  (CSE  494)  Software 
Professional  Development  Program  course.  The  Object  Oriented  Analysis  and  Design 
course  is  a  Professional  Continuing  Education  (PCE)  course  that  is  graded  on  a  pass/fail 
basis.  Students  in  CSE  494  were  required  to  achieve  an  overall  course  grade  of  70%  to 
receive  a  passing  grade. 

Description  of  Educational  Delivery 

The  AFIT/LSS  software  engineering  satellite  courses  are  offered  over  a  10  week 
quarter,  being  held  for  one  and  a  half  hours  a  day,  three  times  a  week.  This  particular 
satellite  course  consisted  of  a  live,  one-way  video  satellite  broadcast  from  the  Air  Force 
Institute  of  Technology  site  located  at  Wright-Patterson  AFB,  to  multiple  downlink  sites 
at  other  Air  Force  bases.  Each  site,  including  the  AFIT  site,  had  a  two-way  audio 
connection  with  push-to-talk  microphones.  These  microphones  allowed  remote  students 
to  communicate  with  the  instructor  and  the  other  students.  The  satellite  classroom  at 
AFIT  was  composed  of  traditional  face-to-face  students  who  also  used  the  push-to-talk 
microphones  to  communicate  with  the  other  sites.  Each  site  recorded  the  satellite 


22 


broadcast  and  distributed  the  videotapes  to  students  who  were  unable  to  attend  class.  The 
remote  sites  that  could  not  receive  the  broadcast  were  sent  copies  of  the  lectures  for 
distribution  to  their  distance  education  students. 

The  AFIT  satellite  classroom  was  equipped  with  multiple  cameras  for  different 
filming  angles,  a  white  board,  overhead  projector,  two  television  monitors  that  displayed 
the  live  broadcast,  push-to-talk  microphones  located  on  each  host  site’s  student  desk,  and 
an  uplink  to  the  remote  sites.  The  remote  classrooms  were  located  at  base  education 
centers  of  other  Air  Force  bases.  Each  remote  classroom  was  equipped  with  a  television 
monitor,  push-to-talk  microphones  for  each  student,  a  video  cassette  recorder  and  a 
satellite  downlink  to  receive  the  AFIT  broadcast. 

Computer-mediated  communication  (CMC)  in  the  form  of  a  moderated  mailing  list 
was  used  to  facilitate  discussion  among  the  remote  students.  To  better  isolate  the  effects 
of  the  CMC  on  the  satellite  students,  it  was  necessary  to  exclude  the  traditional  students 
from  the  mailing  list.  However,  the  traditional  students  were  allowed  to  discuss  issues 
outside  the  class  sessions  with  other  traditional  students  and  the  instructor. 

The  researcher  acted  in  the  capacity  of  the  moderator  with  the  ability  to  subscribe 
students,  unsubscribe  students,  and  forward  all  appropriate  electronic  mail  received  to  all 
of  the  subscribers  on  the  mailing  list.  The  remote  students  were  required  to  subscribe  to 
the  mailing  list  by  the  end  of  the  first  week  of  class,  check  their  electronic  mail  account  a 
minimum  of  three  times  a  week  and  contribute  to  the  mailing  list  at  least  once  every  two 
weeks.  These  mailing  list  requirements  were  detailed  to  the  students  on  the  first  day  of 
class  in  the  handout  located  at  Appendix  B.  The  instructor  sent  out  two  to  three  messages 
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each  week  pertaining  to  the  course,  course  material,  homework  assignment  or  feedback 
requests  on  educational  technology,  instructor  delivery  or  course  content  to  the 
subscribers  of  the  mailing  list. 

The  areas  of  responsibility  for  the  researcher,  instructor,  test  site  administrators, 
course  administrator  and  computer  systems  administrator  and  the  timeframe  of  completion 
for  each  task  for  this  research  project  can  be  found  at  Appendix  C. 


Research  Design 

Since  the  distance  education  group  and  traditional  group  were  not  randomly 
assigned,  this  field  experiment  was  set  up  in  the  nonequivalent  control  group  design,  and 
more  specifically,  an  intact  equivalent  group  design  —  where  both  groups  of  students  were 
assembled  naturally  (Cooper  and  Emory,  1995:368,  Campbell  and  Stanley,  1963:47). 

The  nonequivalent  control  group  design  is  diagrammed  as  follows  (Cooper  and  Emory, 
1995:368): 

Oi  X  O2 

O3  O4 

The  0  refers  to  some  process  of  observation  or  measurement  and  the  X  represents 
the  exposure  of  the  distance  education  group  to  the  experimental  variable  —  educational 
delivery  by  satellite/CMC.  The  Xs  and  Os  in  a  given  row  are  applied  to  the  same  group. 
The  absence  of  the  symbol  R  indicates  that  the  assignment  to  treatment  groups  is 
nonrandom.  The  dashed  line  also  indicates  that  the  two  groups  are  not  equated  by  random 
assignment  (Campbell  and  Stanley,  1963:6). 
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The  distance  education  group,  composed  of  the  students  at  the  remote  sites, 
received  the  live  satellite  broadcast  supplemented  with  computer-mediated 
communication.  The  traditional  group,  composed  of  the  local  Wright-Patterson  students, 
received  the  face-to-face  instruction  that  was  not  supplemented  with  CMC.  The  two 
groups  are  defined  as  follows: 


Distance  education:  A  ‘traditional’  satellite  broadcast  course  to  the 
remote  site  students.  Students  and  instructor  were  geographically  separate. 
Lessons  were  viewed  at  the  remote  base’s  education  center.  Homework 
assignments  were  completed  through  electronic  mail,  facsimile 
transmissions  and  written  correspondence.  Exams  and  surveys  were 
completed  under  the  supervision  of  the  education  center’s  test 
administrator  and  were  sent  to  and  fi’om  the  remote  site  by  certified  mail. 
The  student’s  direct  contact  with  the  instructor  and  other  students  during 
class  was  with  the  push-to-talk  microphones  and  during  non-class  hours, 
contact  was  over  the  mailing  list,  direct  e-mail  or  telephone  contact. 

Traditional:  A  traditional  lecture  course  that  was  modified  for  satellite 
broadcast.  Lessons  were  held  in  the  AFIT  satellite  classroom  and  students 
were  responsible  for  using  the  push-to-talk  microphones  whenever  they 
discussed  items  during  classtime.  Students  had  direct  contact  with  the 
instructor  and  host  site  students  three  times  a  week  during  classtime. 
During  non-class  hours,  students  had  access  to  telephone  and  face-to-face 
contact  with  the  other  students  and  the  instructor.  Exams  and  surveys 
were  administered  by  the  instructor  during  classtime. 


Both  sets  of  students  were  subjected  to  the  same  material,  homework  assignments, 
surveys,  exams  and  instructor  —  ensuring  a  commonality  of  learning  processes  and 
objectives. 
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Population 

The  population  consisted  of  the  Air  Force  Software  Engineering  students  who  met 
the  attendance  prerequisites  as  detailed  in  the  AFIT  School  of  Systems  and  Logistics  PCE 
FY95  catalog.  "Students  must  be  accepted  into  the  Software  Professional  Development 
Program  (SPDP),  which  requires  a  bachelors  degree  with  a  strong  computer  science 
background.  In  addition,  students  must  have  computer  programming  experience  and  a 
basic  knowledge  of  data  structures  and  algorithm  analysis"  (AFIT/LS  PCE  catalog, 
1995:53). 

Sample 

The  sample  consisted  of  the  students  who  voluntarily  enrolled  in  the  AFIT/LSS 
Object  Oriented  Analysis  and  Design  (CSE  494)  software  engineering  course.  The 
students  were  required  to  have  already  taken  the  prerequisite.  Software  Systems 
Engineering  (CSE  492).  The  class  was  in  session  from  10  July  1995  to  22  September 
1995.  The  Object  Oriented  Analysis  and  Design  course  enrolled  51  distance  education 
students  and  12  traditional  face-to-face  students  from  Wright  Patterson  AFB.  The 
distance  education  participants  were  be  the  remote  students  located  at  other  Air  Force 
bases  that  received  the  satellite  broadcast  (19  downlink  sites).  The  remote  sites  consisted 
of  as  many  as  1 1  students  and  as  few  as  one  student. 
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Hypotheses 

This  research  effort  examined  the  performance,  interaction,  and  attitude  outcomes  of 
the  distance  education  group  and  traditional  group  by  testing  five  hypotheses.  The  first 
hypothesis  tested  for  significant  differences  on  performance  scores  between  the  two 
groups.  The  second  and  third  hypotheses  examined  the  two  groups  for  differences  on  the 
student-to-student  interaction  and  student-to-instructor  interaction  that  occurred  during 
non-class  hours.  Hypothesis  four  tested  whether  more  positive  pre-course  attitudes  led  to 
higher  mailing  list  participation  rates  for  the  distance  education  group.  The  last  hypothesis 
(hypothesis  five)  tested  for  significant  differences  between  the  two  groups  on  course  and 
instructor  end-of-course  ratings.  The  five  research  hypotheses  that  were  tested  are  as 
follows: 


Hypothesis  1.  Ho:  There  will  be  no  significant  difference  between  the 
distance  education  and  traditional  students  on  the  scores  measuring  student 
performance. 

Ha:  There  will  be  a  significant  difference  between  the  distance  education 
and  traditional  students  on  the  scores  measuring  student  performance. 

Hypothesis  2.  Ho:  The  distance  education  students  will  not  report  more 
interaction  with  fellow  students  during  non-class  hours  than  the  traditional 
students. 

Ha:  The  distance  education  students  will  report  more  interaction  with 
fellow  students  during  non-class  hours  than  the  traditional  students. 

Hypothesis  3.  Ho:  The  distance  education  students  will  not  report  more 
interaction  with  the  instructor  during  non-class  hours  than  the  traditional 
students. 

Ha:  The  distance  education  students  will  report  more  interaction  with  the 
instructor  during  non-class  hours  than  the  traditional  students. 

Hypothesis  4a:  Ho:  Distance  education  students  with  more  positive  pre¬ 
course  attitudes  towards  computers  will  not  be  more  likely  to  have  higher 
mailing  list  participation  rates. 

Ha:  Distance  education  students  with  more  positive  pre-course  attitudes 
towards  computers  will  be  more  likely  to  have  higher  mailing  list 
participation  rates. 
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Hypothesis  4b:  Ho:  Distance  education  students  with  more  positive  pre¬ 
course  expectations  towards  the  mailing  list  will  not  be  more  likely  to  have 
higher  mailing  list  participation  rates. 

Ha!  Distance  education  students  with  more  positive  pre-course 
expectations  towards  the  mailing  list  will  be  more  likely  to  have  higher 
mailing  list  participation  rates. 

Hypothesis  5a:  Ho:  There  will  be  no  significant  difference  between  distance 
education  and  traditional  students  on  the  post-course  survey  instructor 
ratings. 

Ha:  There  will  be  a  significant  difference  between  distance  education  and 
traditional  students  on  the  post-course  survey  instructor  ratings. 

Hypothesis  5b:  Ho:  There  will  be  no  significant  difference  between 
distance  education  and  traditional  students  on  the  post-course  survey 
course  ratings. 

Ha:  There  will  be  a  significant  difference  between  distance  education  and 
traditional  students  on  the  post-course  survey  instructor  ratings. 


Data  Collection  Methodology 

Research  Instruments.  In  order  to  evaluate  the  above  hypotheses,  data  was  collected 
from  each  student  using  the  following  pre-and  post-course  research  instruments:  pretest, 
midterm  exam,  final  exam,  overall  course  grade,  pre-course  survey,  post-course  survey 
and  participation  rates. 

Performance  was  measured  by  the  grades  received  on  the  pretest,  the  midterm  exam, 
the  final  exam  and  the  overall  course.  Mailing  list  participation  rates  for  distance 
education  students  were  computed  by  the  number  of  postings  that  were  contributed  by 
each  student  during  the  duration  of  the  course  (Appendix  D). 

Student  demographic  data  was  collected  on  the  pre-course  survey  (Appendix  E). 
Some  of  the  student  characteristics  that  were  collected  include:  age,  sex,  present  job  title. 
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time  in  present  job,  time  working  in  the  software  engineering  field,  current  course  load, 
experience  with  computers,  number  of  courses  in  software  engineering  and  level  of 
education.  An  explanation  of  how  some  of  the  demographic  data  were  coded  can  be 
found  at  Appendix  F. 

The  pre-course  survey  also  posed  Likert-type  scale  questions  pertaining  to  attitudes 
towards  computers,  computer  experience,  and  connectivity  issues.  In  addition,  the  pre¬ 
course  survey  collected  the  distance  education  group’s  expectations  for  the  mailing  list. 

The  post-course  survey  (Appendix  G)  posed  Likert-type  scale  questions  pertaining 
to  time  spent  on  the  course,  experience  with  the  mailing  list,  time  spent  using  the  mailing 
list,  the  course  and  the  instructor  rating.  The  post-course  survey  also  collected  data  on 
experience  with  the  mailing  list  for  the  distance  education  group. 

To  protect  the  student's  privacy,  names  were  not  necessary  for  this  research. 
However,  the  last  four  digits  of  the  students’  social  security  numbers  were  used  to  match 
the  results  of  the  pre-  and  post-course  research  instruments.  Reftisal  to  complete  any 
survey,  under  the  protection-of-human  rights  subjects'  regulations,  did  not  affect  the 
student’s  grades  in  any  way. 

Instrument  Construction.  The  pretest,  midterm  exam,  and  final  exam  were 
constructed  and  graded  by  the  instructor.  The  pretest,  designed  specifically  by  the 
instructor  for  this  experiment,  was  used  to  establish  a  baseline  performance  score  for  each 
student.  An  overall  course  grade  for  each  student  was  determined  by  the  instructor  fi'om 
his  or  her  test  and  homework  scores. 


29 


The  pre-  and  post-course  surveys  were  constructed  by  the  researcher.  Questions  on 
demographics  (#1-21)  were  developed  by  the  researcher.  Attitudinal  questions  (pre¬ 
course  #22-39,  post-course  #4-37)  were  developed  by  Starr  Roxanne  Hiltz  for  the 
evaluation  of  the  Virtual  Classroom  (registered  trademark)  found  in  Chapter  7  (p.l34)  of 
Online  Education:  Perspectives  on  a  New  Environment,  edited  by  Linda  M.  Harasim, 
1991.  These  questions  were  used  with  permission,  as  given  on  page  134. 

Both  pre-course  and  post-course  surveys  were  evaluated  by  software  engineering 
instructors,  as  well  as  AFIT  graduate  faculty  members,  for  content  and  research  suitability. 

Instrument  Administration.  The  pretest  and  pre-course  survey  were  administered  on 
the  second  day  of  class  by  the  instructor  for  the  traditional  group,  and  by  the  base 
education  centers’  test  administrators  for  the  distance  education  group.  The  final  and  post¬ 
course  survey  was  administered  on  the  last  day  of  class.  The  distance  education  group's 
survey  and  test  instruments  were  sent  to  and  from  the  education  centers  by  certified  mail. 

Data  Analysis 

The  performance,  interaction,  and  attitude  outcomes  served  as  the  dependent 
variables  for  this  study.  The  dependent  variables,  the  independent  variables  and  how  each 
were  specifically  measured  can  be  found  at  Appendix  H.  In  general,  the  first  dependent 
variable,  performance  outcome,  was  measured  by  grades  from  the  pretest,  the  midterm 
and  final  exams,  and  the  overall  course.  The  second  dependent  variable,  interaction 
outcome,  was  measured  by  the  responses  given  on  the  Likert-type  post-course  survey 
interaction  questions.  The  final  dependent  variable,  attitude  outcomes,  was  measured  by: 
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1)  the  pre-course  survey  of  student's  attitudes  toward  the  computer,  the  course,  and  the 
delivery  medium;  and  2)  the  post-course  survey  of  the  student's  attitudes  toward  the 
computer,  course  content,  course  delivery,  instructor,  and  course. 

Much  consideration  was  given  to  Cooper  and  Emory’s  discussion  of  evaluating 
ordinal  scales  (Cooper  and  Emory,  1995:145).  Cooper  and  Emory  detail  the  differences 
of  opinion  among  behavioral  scientists  on  using  parametric  over  nonparametric  methods  to 
evaluate  ordinal  data  found  in  Likert-type  questionnaires.  “One  position  is  that  the  use  of 
parametric  tests  is  incorrect  on  both  theoretical  and  practical  grounds...  At  the  other 
extreme,  some  behavioral  scientists  argue  that  parametric  tests  are  usually  acceptable  for 
ordinal  scales”  (Cooper  and  Emory,  1995;  145).  There  is  also  a  view  between  these  two 
extremes  that  recognizes  there  are  risks  --  but  these  risks  are  usually  not  significant. 
Kerlinger,  one  of  Cooper  and  Emory’s  contributors,  states  that  the  best  procedure  is  to 
“treat  ordinal  measurements  as  though  they  were  interval  measurements  but  to  be 
constantly  alert  to  the  possibility  of  gross  inequality  of  intervals”  (Cooper  and  Emory, 

1995  :145).  In  light  of  these  opinions,  this  researcher  decided  to  use  parametric  statistical 
tests  on  all  five  hypotheses. 

All  dependent  variables,  including  performance,  interaction,  and  attitudes  were 
analyzed  by  either  a  one-way  analysis  of  variance  (ANOVA)  or  a  Student’s  t-Test.  All 
tests  were  evaluated  at  a  95%  confidence  interval  (alpha  =  .05). 

Demographic  data  on  the  pre-course  survey  was  used  to  compare  the  homogeneity 
of  the  two  groups.  The  demographic  data  for  each  question  was  grouped  into  categories, 
and  each  category  was  assigned  an  ordinal  ranking  for  purposes  of  comparison.  A  one- 
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way  ANOVA  was  completed  on  each  demographic  question  to  determine  if  there  was  a 
significant  difference  between  the  two  groups.  A  similar  test  on  attitudes  towards 
computers  was  also  conducted  on  the  two  groups.  Only  those  students  who  completed  a 
pre-course  survey  were  considered  in  this  test. 

Performance  scores  for  each  test  and  overall  grade  was  analyzed  by  a  one-way 
ANOVA  between  the  two  groups.  Only  those  students  completing  both  the  pretest  and 
posttest  measures  were  included  in  this  Student’s  t-Test  analysis. 

Interaction  outcomes  were  compared  between  the  two  groups  by  conducting  a  one- 
tailed  Student’s  t-Test  on  the  post-course  survey  questions  on  student-to-student  and 
student-to-instructor  interaction. 

A  one-tailed  Student’s  t-Test  was  also  conducted  on  the  distance  education  group  to 
determine  if  students  with  more  positive  pre-course  attitudes  towards  computers  and  the 
mailing  list  had  higher  participation  rates. 

Student  attitudes  on  the  instructor  and  course  were  analyzed  by  a  one-way  ANOVA 
for  each  post-course  survey  question  for  significant  differences  between  the  two  groups. 

A  two-tailed  Student’s  t-Test  was  conducted  on  the  combined  instructor  and  course 
ratings  to  test  for  significant  differences  between  the  two  groups. 


Summary 

This  chapter  covered  the  description  of  the  educational  delivery,  the  general  research 
approach,  the  hypotheses  intended  to  be  tested,  the  population  and  sample  of  subjects  to 
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be  studied,  and  the  data  collection  methodology  and  data  analysis  methods  that  were  used 
to  test  the  hypotheses.  The  next  chapter  will  present  the  results  and  analysis  of  the 
experiment. 
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IV.  Results  and  Analysis 


Introduction 

This  chapter  presents  the  descriptive  statistics  and  corresponding  statistical  test 
statistics  for  the  data  and  hypotheses  described  in  the  methodology  chapter. 

The  dependent  variables  analyzed  in  this  study  were  performance,  interaction,  and 
attitude  outcomes.  The  first  dependent  variable,  performance  outcome,  was  measured  by 
grades  from  the  pretest,  the  midterm  and  final  exams,  and  the  overall  course  average.  The 
second  dependent  variable,  interaction  outcome,  was  measured  by  the  responses  given  on 
the  post-course  survey  interaction  questions.  The  final  dependent  variable,  attitude 
outcomes  was  measured  by  responses  from  the  pre-  and  post-course  survey  and  mailing 
list  participation  rates. 

Descriptive  statistics  (demographic  data  and  attitudes  towards  computers)  from  the 
pre-course  survey  were  used  to  compare  the  homogeneity  of  the  two  groups.  A  one-way 
analysis  of  variance  (ANOVA)  was  completed  on  each  question  to  determine  if  there  was 
a  significant  difference  between  the  distance  education  and  traditional  groups.  Only  those 
students  who  completed  the  pre-course  survey  were  included  in  this  analysis. 

All  dependent  variables  were  analyzed  by  either  a  one-way  analysis  of  variance  or  a 
Student’s  t-Test.  All  statistical  tests  were  conducted  at  a  confidence  level  of  95%  (alpha 
=  .05).  Data  was  analyzed  using  the  Microsoft  EXCEL  (registered  trademark) 

I  spreadsheet  package. 
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There  were  sixty-three  students  enrolled  in  the  course  for  Professional  Continuing 
Education  (PCE)  credit  —  twelve  were  from  Wright-Patterson  and  fifty-one  were  distance 
education  students.  Eleven  students  (all  distance  education)  withdrew  from  the  course. 

At  the  time  of  this  writing,  two  distance  education  students  had  incomplete  final  and 
overall  course  grades  due  to  extensive  travel  scheduling  problems  that  kept  them  from 
completing  the  course. 

Descriptive  Statistics 

Demographic  Data.  A  one-way  analysis  of  variance  was  conducted  on  each 
demographic  question  to  determine  if  the  distance  education  and  traditional  groups  were 
significantly  different.  This  test  was  conducted  to  compare  the  homogeneity  of  the  two 
groups.  Only  those  students  who  completed  a  pre-course  survey  were  considered  in  this 
test  (25  from  the  distance  education  and  10  from  the  traditional  group).  The  results  of 
the  one-way  ANOVA  are  in  Table  1. 

The  ANOVA  revealed  the  two  groups  were  not  significantly  different,  except  on 
two  aspects  —  education  level  and  self-rated  computer  proficiency.  In  both  cases,  the 
traditional  group  scored  higher  —  meaning  the  traditional  group  had  higher  educational 
levels  and  higher  computer  proficiency  self-ratings  compared  to  the  distance  education 
group. 
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Table  1 


[  Demographic  Data  | 

Variables 

Distance  Education 

Traditional 

ANOVA 

n 

Mean 

S.D. 

n 

Mean 

S.D. 

Sig.ofF 

Sig 

1.  SEX  (1=male,  2=female) 

25 

1.12 

0.331 

10 

1.1 

0.316 

0.8714 

2.  AGE  (#  of  years) 

25 

34.4 

■Q 

0.8942 

3.  MILITARY  (1=yes.2=no) 

25 

1.68 

0.4761 

10 

1.7 

0.483 

0.9116 

4.  RANK/GRADE  (1=JR,2=SR) 

25 

1.76 

0.4359 

10 

1.9 

0.316 

0.3643 

7.  MONTHS  IN  CURRENT  JOB 

m 

IB 

38 

34.467 

0.6985 

8.  S/W  ENGR  JOB  (1=yes.  2=no) 

m 

IB 

1 

0 

0.3718 

9.  LENGTH  IN  S/W  FIELD  (Months) 

m 

IB 

94.7 

73.43 

0.5362 

10.  ED  LEVEL  (1=  undergrad,  2=some  grad,  3=grad) 

m 

HIS 

IB 

0.7071 

* 

11.#  UNDERGRAD  S/W  ENGR  CREDITS 

R! 

m 

12.  #  GRAD  S/W  ENGR  CREDITS 

183 

m 

BUEn 

13.  #  PCE  S/W  ENGR  COURSES 

2.652 

1.7992 

9 

1.2247 

14.  TIME  EXPECTED  (1  =<1  hr,2=1 -2hr,3=2-3hr,4=>3hr) 

m 

3.208 

0.8836 

9 

2.667 

0.7071 

17.  COMPUTER  PROFICIENCY(1=no  exp.  to  5=very) 

24 

3.958 

0.8587 

9 

4.778 

0.1944 

* 

20.  USE  E-MAIL  (1  =frequently  to  7=never) 

m 

1.96 

1.5132 

9 

1.222 

0.6663 

0.1701 

21 .  MAILING  LIST  MEMBER  (1=yes.  2=no) 

25 

1.48 

0.6 

9 

1.111 

0.3333 

0.0529 

Attitudes  Towards  Computers  Data.  In  order  to  establish  a  similarity  between  the 
two  groups  on  attitudes  toward  computers,  the  pre-course  survey  data  on  attitudes 
towards  computers  was  analyzed  using  a  one-way  ANOVA  between  the  distance 
education  group  and  traditional  group.  The  results  listed  in  Table  2  indicate  no  significant 
differences  between  the  groups  on  attitudes  towards  computers. 


Table  2 


I  Attitudes  Towards  Computers  Data  | 

Variables 

Distance  Education 

Traditional 

ANOVA 

Sig 

n 

Mean 

S.D. 

n 

Mean 

S.D. 

Sig.  of  F 

22.  STIMULATING  VS  DULL 

25 

2.2 

1.041 

9 

2.111 

1.05 

0.828 

23.  FUN  VS  DREARY 

25 

2.2 

1.08 

9 

1.77778 

0.66666 

0.28222 

24.  EASY  VS  DIFFICULT 

25 

2.52 

1.388 

9 

1 .88889 

0.78174 

0.20822 

25.  PERSONAL  VS  IMPERSONAL 

25 

3.44 

1.73397 

9 

3.66667 

1.73205 

0.73879 

26.  HELPFUL  VS  HINDERING 

25 

2.12 

1.20138 

9 

2.22222 

1.3017 

0.83169 

27.  NON  THREATENING  VS  THREATENING 

25 

1.96 

1.17189 

9 

2.55556 

1.424 

0.22553 

28.  EFFICIENT  VS  INEFFICIENT 

25 

2.4 

1.354 

9 

2.22222 

0.66667 

0.71003 

29.  OBLIGING  VS  DEMANDING 

25 

3.28 

1.54164 

9 

4.11111 

1.53659 

0,17473 

30.  DESIRABLE  VS  UNDESIRABLE 

25 

1,84 

1.02794 

9 

1.88889 

1.05409 

0.904 
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Performance  Data 


Performance  scores  for  each  exam  and  overall  course  grade  were  analyzed  by  a  one¬ 
way  ANOVA  between  the  two  groups.  The  grades  received  on  the  instructor- 
developed  exams  were  used  to  test  the  following  hypothesis; 

Hypothesis  1.  Ho:  There  will  be  no  significant  difference  between  the 
distance  education  and  traditional  students  on  the  scores  measuring  student 
performance. 

Ha!  There  will  be  a  significant  difference  between  the  distance  education 
and  traditional  students  on  the  scores  measuring  student  performance. 


The  results  of  the  one-way  ANOVA  test  on  the  grades  for  the  traditional  and 
distance  education  group  are  below  in  Table  3. 


Table  3 


1  Grade  Data  | 

Variables 

Distance  Education 

Traditional 

ANOVA 

Sig 

n 

Mean 

S.D. 

n 

Mean 

S.D. 

Sig.  ofF 

Pretest 

28 

55.35 

20.146 

11 

76.66 

18.76 

* 

Midterm 

40 

83 

10.178 

12 

89.167 

5.967 

Final 

38 

68.2 

14.9 

12 

77.917 

8.8468 

* 

Overall 

38 

83.29 

12.36 

12 

91.75 

6.312 

* 

The  null  hypothesis  for  performance  was  rejected  for  the  pretest,  final  exam  and 
overall  course  grades  (significance  of  F  was  less  than  alpha  (.05)).  There  was  a  significant 
difference  between  the  distance  education  and  traditional  students  on  the  scores  measuring 
student  performance  on  the  pretest,  final  exam  and  overall  course  grade.  However,  there 
was  no  significant  difference  between  the  distance  education  and  traditional  groups  on  the 
midterm  exam  scores  (significance  of  F  was  not  less  than  alpha  (.05)). 
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Interaction  Data 


Interaction  outcomes  were  compared  between  the  traditional  and  distance  education 
groups  by  conducting  a  one-tailed  Student’s  t-Test  on  the  post-course  survey  questions  on 
student-to-student  and  student-to-instructor  interaction. 

Student-to-student  interaction  during  non-class  hours  was  compared  using  the  post¬ 
course  survey  question  #2.  Question  #2  was  a  6-point  Likert-type  question  that  asked 
“How  often  did  you  communicate  with  other  students  outside  of  class,  by  computer, 
‘face-to-face’  or  on  the  telephone?”  The  possible  choices  were  as  follows;  1-never,  2- 
rarely,  3-sometimes,  4-often,  5-frequently,  6-constantly.  The  following  hypothesis  was 
tested: 


Hypothesis  2,  Ho:  The  distance  education  students  will  not  report  more 
interaction  with  fellow  students  during  non-class  hours  than  the  traditional 
students. 

Ha:  The  distance  education  students  will  report  more  interaction  with 
fellow  students  during  non-class  hours  than  the  traditional  students. 


The  two  groups  were  also  compared  on  student-to-instructor  interaction.  Post¬ 
course  survey  question  #3  was  used  to  test  the  following  hypothesis: 


Hypothesis  3.  Ho:  The  distance  education  students  will  not  report  more 
interaction  with  the  instructor  during  non-class  hours  than  the  traditional 
students. 

Ho:  The  distance  education  students  will  report  more  interaction  with  the 
instructor  during  non-class  hours  than  the  traditional  students. 


The  results  of  the  one-tailed  Student’s  t-Tests  for  hypothesis  2  and  3  are  listed 
below  in  Table  4. 


Table  4 
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1  Student's  t-Test  Results  of  Student-to-Student  and  Student-to-lnstructor  Interaction  | 

Variables 

n 

Mean 

S.D. 

t-value 

d.f. 

1 -tailed  prob 

significant 

Student-to-Student 

Distance  Education 

24 

2.8 

1.279 

Traditional 

10 

3.375 

0.9189 

-1.2851 

35 

0.103999 

Student-to-lnstructor 

Distance  Education 

24 

2.41667 

1.018 

Traditional 

10 

2 

0.6667 

-1.187 

32 

0.121984 

Both  null  hypotheses  2  and  3  were  not  rejected.  There  was  no  significant  difference 
in  student-to-student  or  student-to-instructor  interaction  rates  between  the  two  groups. 


Attitude  Data 

A  one-tailed  Student’s  t-Test  was  conducted  on  the  distance  education  group  to 
determine  if  students  with  more  positive  pre-course  attitudes  towards  computers  and  the 
mailing  list  had  higher  mailing  list  participation  rates.  It  was  hypothesized  that  student’s 
with  more  positive  pre-course  attitudes  about  computers  and  the  educational  medium  will 
have  higher  mailing  list  participation  rates  and  be  more  accepting  of  the  mailing  list 
technology  and  thus,  participate  more.  The  pre-course  attitudes  towards  computers 
questions  (#22-30)  and  pre-course  expectations  for  the  mailing  list  (questions  #31-38) 
were  used  in  evaluating  the  following  hypotheses: 

Hypothesis  4a:  Ho:  Distance  education  students  with  more  positive  pre¬ 
course  attitudes  towards  computers  will  not  be  more  likely  to  have  higher 
mailing  list  participation  rates. 
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Ha!  Distance  education  students  with  more  positive  pre-course  attitudes 
towards  computers  will  be  more  likely  to  have  higher  mailing  list 
participation  rates. 

Hypothesis  4b:  Ho:  Distance  education  students  with  more  positive  pre¬ 
course  expectations  towards  the  mailing  list  will  not  be  more  likely  to  have 
higher  mailing  list  participation  rates. 

Ha!  Distance  education  students  with  more  positive  pre-course 
expectations  towards  the  mailing  list  will  be  more  likely  to  have  higher 
mailing  list  participation  rates. 

The  distance  education  group’s  responses  to  the  pre-course  survey  questions  were 
totaled  and  averaged  for  each  student  (7-point  Likert-type  questions,  1  being  positive  and 
7  being  negative).  The  averages  for  the  group  were  analyzed  and  divided  into  more 
positive  and  less  positive  groups  by  the  researcher.  The  students  with  the  most  positive 
averages  were  included  in  the  more  positive  group  and  the  students  with  the  least  positive 
averages  were  included  in  the  less  positive  group.  The  group  sizes  are  unequal  because 
the  more  positive  students  had  several  averages  with  the  same  value,  making  the  division 
line  between  the  two  groups  difficult  to  ascertain.  The  students  in  the  more  positive  pre¬ 
course  computer  attitudes  group  had  averages  between  1  and  2.333  and  the  students  in 
the  less  positive  pre-course  computer  attitudes  group  had  averages  between  2.556  and 
3 .44.  The  students  with  the  more  positive  pre-course  expectations  towards  the  mailing 
list  had  averages  from  1.42  to  3.71.  While  the  students  with  more  negative  pre-course 
expectations  towards  the  mailing  list  had  averages  from  3.857  to  6.16.  The  number  of 
mailing  list  postings  for  the  more  positive  and  less  positive  groups  were  compared  and 
analyzed  with  a  one-tailed  Student’s  t-Test.  Results  of  that  test  are  located  in  Table  5, 
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Mailing  list  participation  rates  for  distance  education  students  were  computed  by  the 
number  of  postings  that  were  contributed  by  each  student  during  the  duration  of  the 
course.  Some  students  did  not  participate  at  all  (count  of  0),  while  others  had  4  or  5 
contributions. 


Table  5 


1  Student’s  t-Test  Results  of  Pre-course  Attitudes  and  Mailing  List  Participation  | 

Variables 

n 

Mean 

S.D. 

t-value 

d.f. 

1 -tailed  prob 

significant 

Computer  Attitudes  (4a) 

More  Positive 

m 

1.385 

1.6602 

Less  Positive 

10 

1.2 

1.2293 

0.2944 

21 

0.385672 

Expectations  of  Mailing  List  (4b) 

More  Positive 

13 

1.5 

1.6132 

More  Negative 

10 

1 

1,2472 

0.87228 

21 

0.19646 

The  null  hypothesis  was  not  rejected  for  either  hypothesis  4a  or  hypothesis  4b  (1- 
tailed  probability  is  not  less  than  alpha  (.05)).  There  was  no  significant  difference  between 
the  two  groups  --  more  positive  pre-course  attitudes  towards  computers  and  more  positive 
pre-course  expectations  towards  the  mailing  list  did  not  seem  to  effect  mailing  list 
participation  rates. 

In  an  effort  to  determine  whether  the  treatment  affected  the  attitudes  about  the 
mailing  list  for  the  distance  education  group,  the  following  (unnumbered)  hypothesis  was 
tested  using  a  one-way  analysis  of  variance: 

Hypothesis:  Ho:  There  will  be  no  significant  difference  between  the  pre-course 
expectation  about  using  the  mailing  list  and  post-course  experience  with  the 
mailing  list. 
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Ha!  There  will  be  a  significant  difference  between  the  pre-course  expectation 
about  using  the  mailing  list  and  post-course  experience  with  the  mailing  list. 

Results  from  the  one-way  ANOVA  are  located  in  Table  6  below. 


Table  6 


1  Expectations  vs  Experience  with  Mailing  List  | 

Variables 

Pre-course 

Post-course 

ANOVA 

Sig 

n 

Mean 

S.D. 

n 

Mean 

S.D. 

Sig.  of  F 

Pre31/Post28  EASY  VS  HARD  TO  LEARN 

23 

2.3913 

1.3396 

24 

1.79167 

0.8836 

0.075577 

Pre32/Post29  FRIENDLY  VS  IMPERSONAL 

23 

2.826 

1.2668 

24 

2.875 

1.5691 

0.90716 

Pre33/Post30  NOT  FRUSTRATING  VS  FRUSTRATING 

25 

3.4 

1.9149 

24 

3.125 

1.8954 

0.615881 

Pre34/Post31  PRODUCTIVE  VS  NOT  PRODUCTIVE 

25 

3.68 

1.9305 

24 

4.16667 

1.7611 

0.306187 

Pre35/Post32  MAILING  LIST  AND  EFFICIENCY 

25 

4.08 1 

KBltlrZl 

m 

4.95833 

1.6545 

0.065754 

Pre36/Post33  MAILING  LIST  AND  QUALITY 

24 

m 

4.95833 

1.8528 

0.073145 

Pre38/Post37  OVERALL  MAILING  LIST 

24 

El 

4.5 

1.6681 

0.344221 

The  results  of  the  one-way  analysis  of  variance  indicate  that  there  were  no  significant 
differences  between  the  mailing  list  expectations  and  the  mailing  list  experiences  with  the 
distance  education  group  at  the  95%  confidence  level.  The  null  hypothesis  was  not 
rejected  for  any  of  the  variable  questions. 

The  last  attitudinal  hypotheses  to  be  tested  dealt  with  instructor  and  course  ratings. 
This  research  effort  was  interested  in  testing  whether  the  groups  differed  in  their  ratings 
on  the  course  and  the  instructor.  Student  attitudes  on  the  instructor  and  course  were 
analyzed  by  a  one-way  ANOVA  for  each  post-course  survey  question  for  significant 
differences  between  the  two  groups.  A  two-tailed  Student’s  t-Test  was  conducted  on  the 
combined  instructor  and  course  ratings  to  test  for  significant  differences  between  the  two 
groups.  The  following  hypotheses  were  tested  using  a  one-way  ANOVA  (results  are 
found  below  in  Table  7  and  8)  on  the  5-point  Likert-type  questions  regarding  instructor 
and  course  ratings  (1 -strongly  agree  to  5 -strongly  disagree); 
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Hypothesis  5a:  Ho:  There  will  be  no  significant  difference  between  distance 
education  and  traditional  students  on  the  post-course  survey  instructor 
ratings. 


Ha!  There  will  be  a  significant  difference  between  distance  education  and 
traditional  students  on  the  post-course  survey  instructor  ratings. 

Hypothesis  5b:  Hq:  There  will  be  no  significant  difference  between 
distance  education  and  traditional  students  on  the  post-course  survey 
course  ratings. 


H*:  There  will  be  a  significant  difference  between  distance  education  and 
traditional  students  on  the  post-course  survey  instructor  ratings. 


Table  7 


Instructor  Ratings  Data  | 

Variables 

Distance  Education 

Traditional 

A  NOVA 

Sig 

n 

Mean 

S.D. 

n 

Mean 

S.D. 

Sig.  of  F 

7.  INSTRUCTOR  ORGANIZED  COURSE  WELL 

24 

2.5 

1.02151 

10 

2.4 

0.96609 

0.79344 

8.  GRADINGWAS  FAIR  AND  IMPARTIAL 

24 

2.5417 

0.88363 

10 

2.2 

0.63246 

0.27699 

9.  INSTRUCTOR  SEEMS  TO  ENJOY  TEACHING 

24 

2.0833 

0.71728 

10 

2.1 

0.87559 

0.95421 

10.  INSTRUCTOR  HAS  SUFFICIENT  KNOWLEDGE 

24 

2.0417 

0.75062 

10 

1.8 

0.63246 

0.37874 

1 1 .  ENCOURAGED  TO  EXPRESS  IDEAS 

24 

1.75 

0.74399 

10 

0.67494 

0.86277 

12.  MATERIAL  PRESENTED  CLEARLY  ; 

24 

2.125 

0.67967 

10 

1.05934 

0,56748 

13.  INSTRUCTOR  DISCUSSED  OTHER  POVs 

24 

2.4167 

0.97431 

10 

0.82156 

14.  ABLE  TO  GET  PERSONAL  HELP 

24 

2.375 

1.05552 

10 

mm 

0.56764 

15.  MATERIAL  PRESENTED  INTERESTINGLY 

24 

2.5833 

0.97431 

10 

1.03279 

16.  WORK  CRITIQUED  CONSTRUCTIVELY 

24 

2.9583 

1.29297 

10 

0.69921 

17.  OVERALL  RATING  OF  INSTRUCTOR 

24 

2.5 

0.88465 

10 

Table  8 
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1  Course  Ratings  Data 

Variables 

Distance  Education 

Traditional 

ANOVA 

Sig 

n 

Mean 

S.D. 

n 

Mean 

S.D. 

S^.  of  F 

18.  COURSE  MADE  ME  MORE  INTERESTED 

24 

2.4167 

1.100006 

10 

2.1 

0.73786 

0.41165 

19.  LEARNED  A  GREAT  DEAL 

24 

2 

0.93251 

10 

1.8 

0.42164 

0.5224 

20.  GAINED  GOOD  UNDERSTANDING 

24 

2.2083 

0.977093 

10 

1.9 

0.56765 

0.35962 

21 .  LEARNED  TO  IDENTIFY  CENTRAL  ISSUES 

24 

2.25 

0.989071 

10 

2.1 

0.73786 

0.66959 

22.  COMMUNICATE  CLEARLY 

24 

2.375 

0.969698 

10 

2 

0.66666 

0.27384 

23.  COURSE  WAS  A  GOOD  USE  OF  TIME 

24 

2.2917 

0.954584 

10 

1.7 

0.67495 

0.08518 

24.  DEVELOPED  NEW  FRIENDSHIPS 

24 

2.875 

1.153916 

10 

3.4 

0.9669 

0.21569 

25.  LEARNED  TO  VALUE  OTHER  POVs 

24 

2.4583 

0.867754 

10 

2.4444 

0.72648 

0.96817 

26.  OVERALL  COURSE  RATING 

24 

2.66667 

0.816489 

10 

2.3 

0.67495 

0.22033 

The  results  from  all  of  the  ANOVA  tests  indicate  that  there  were  no  significant 
differences  on  the  instructor  ratings  or  course  ratings  between  the  distance  education  and 
traditional  groups.  The  recommendation  of  these  tests  was  to  not  reject  the  null 
hypothesis  for  hypothesis  5a  or  5b. 


Summary 

This  chapter  presented  the  descriptive  statistics  and  corresponding  statistical  test 
statistics  for  the  data  and  hypotheses  described  in  the  previous  chapter.  All  statistical  tests 
were  conducted  at  a  confidence  level  of  95%  (alpha  =  .05).  It  was  discovered  that  there 
were  no  significant  differences  on  the  descriptive  statistics  (demographic  data  and 
computer  attitudes)  between  the  two  distance  education  and  traditional  groups  with  only 
two  exceptions  --  educational  level  and  self-rated  computer  proficiency.  (The  traditional 
group  rated  higher  on  both.)  There  appeared  to  be  a  significant  difference  between  the 
groups  on  the  pretest,  final  exam  and  overall  course  grades,  but  not  on  the  midterm  exam 
grades.  On  the  basis  of  student-to-student  interaction  and  student-to-instructor 
interaction,  no  significant  differences  between  the  two  groups  were  noted.  When  testing 
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the  distance  education  group  for  positive  pre-course  attitudes  and  participation  rates,  it 
was  discovered  that  there  was  no  significant  difference  between  positive  and  negative 
attitudes  and  participation  rates.  The  tests  also  showed  that  the  treatment  did  not  affect 
the  attitudes  towards  the  mailing  list  for  the  distance  education  group.  On  the  last 
hypothesis,  no  significant  difference  on  the  instructor  and  course  ratings  was  noted 
between  the  distance  education  and  traditional  group. 

The  final  chapter  of  this  thesis  will  draw  conclusions  and  recommendations  fi'om  the 
above  results  and  analysis. 
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V.  Conclusions  and  Recommendations 


Introduction 

Many  educators  recognize  that  the  biggest  drawback  to  satellite  distance  education 
is  the  lack  of  interaction  capabilities  for  the  remote  student.  As  educational  institutions 
attempt  to  overcome  this  drawback,  some  are  integrating  computer-mediated 
communication  (CMC)  with  the  satellite  distance  educational  delivery.  This  research 
effort  was  designed  to  analyze  the  comparative  effectiveness  of  an  Air  Force  Institute  of 
Technology  software  engineering  CMC  supplemented  satellite  distance  education  course 
to  the  traditional  educational  delivery  method. 

This  chapter  summarizes  the  conclusions  drawn  fi-om  the  analysis  performed  in  the 
previous  chapter,  in  effect  answering  the  research  questions  set  forth  by  this  research 
project.  Limitations  of  the  research,  recommendations  for  further,  and  a  summary  of  the 
research  effort  are  also  presented  in  this  chapter. 

Conclusions  of  Results  and  Analysis 

Descriptive  Statistics.  The  one-way  ANOVA  test  for  each  demographic  question 
revealed  that  there  were  significant  differences  between  the  two  groups  on  educational 
level  and  self-rated  computer  proficiency.  The  traditional  group  rated  higher  on  both 
accounts.  This  is  not  surprising  because  Wright-Patterson  is  a  highly  technical  base,  and 
the  nature  of  the  jobs  require  more  educational  experience  (and  possibly  more  computer 
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experience)  on  the  whole,  when  compared  to  other  bases.  The  significant  differences  on 
these  items  could  explain  some  of  the  observed  differences  on  performance  outcomes. 

On  attitudes  toward  computers,  it  was  hypothesized  that  there  would  be  no 
significant  differences  between  the  distance  education  and  traditional  groups.  A  one-way 
ANOVA  test  between  the  distance  education  group  and  traditional  group  on  the  pre¬ 
course  survey  on  attitudes  towards  computers  indicated  that  there  were  no  significant 
differences  between  the  groups.  Due  to  the  nature  of  this  course  (computer  oriented), 
these  results  do  not  come  as  a  surprise.  If  there  had  been  a  significant  difference  between 
the  two  groups,  with  the  traditional  group  rating  higher,  it  could  be  hypothesized  that 
differences  in  computer  attitudes  affected  performance  in  the  course.  However,  this  was 
not  the  case  in  this  experiment. 

Performance  Data.  After  conducting  a  one-way  ANOVA  on  the  grade  data,  it  was 
discovered  that  there  was  a  significant  difference  between  the  groups  on  the  pretest,  final 
and  overall  course  grades,  but  not  on  the  midterm  grades.  The  significant  differences  can 
be  attributed  to  the  differences  in  education  level  and  self-assessed  computer  proficiency 
between  the  two  groups.  Since  the  traditional  group  had  higher  education  levels  and  rated 
themselves  higher  on  computer  proficiency,  it  seems  logical  that  they  would  score  higher 
than  the  distance  education  group.  However,  this  assessment  does  not  explain  the 
midterm  grades.  Because  there  was  no  significant  difference  between  the  traditional  and 
distance  education  group  on  the  midterm,  one  cannot  automatically  assume  that  different 
educational  backgrounds  will  lead  to  differences  in  test  scores.  However,  the  lack  of 
difference  between  the  two  groups  can  be  explained  by  other  reasons  —  such  as  leniency  in 
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the  instructor’s  grading  procedures  (i.e.,  the  instructor  curved  grades  to  boost  the  overall 
performance  of  the  class). 

Interaction  Data.  When  the  student-to-student  interaction  and  student-to-instructor 
interaction  rates  outside  the  classroom  were  tested  with  a  Student’s  t-Test,  it  was 
discovered  that  there  were  no  significant  differences  between  the  two  groups.  With  no 
significant  differences  between  the  two  groups,  it  can  be  hypothesized  that  the  mailing  list 
enabled  the  distance  education  group  to  achieve  comparative  interaction  rates  to  the 
traditional  group.  The  mailing  list  provided  the  distance  education  students  with  an 
opportunity  to  interact  with  other  students  and  the  course  instructor. 

Attitude  Data.  When  testing  the  hypothesis  that  more  positive  pre-course  attitudes 
towards  computers  led  to  higher  participation  rates  among  the  distance  education 
students,  it  was  discovered  that  there  were  no  significant  differences  between  positive  and 
negative  computer  attitudes  on  mailing  list  participation  rates.  The  Student’s  t-Test  also 
revealed  that  there  were  no  significant  differences  between  positive  and  negative 
expectations  of  the  mailing  list  and  mailing  list  participation  rates.  Another  test  revealed 
that  the  treatment  did  not  affect  the  attitudes  about  the  mailing  list  for  the  distance 
education  group  —  there  was  no  significant  difference  between  the  pre-course  expectation 
about  using  the  mailing  list  and  post-course  experience  with  the  mailing  list.  It  would 
have  been  more  favorable  to  show  that  the  treatment  positively  affected  attitudes  toward 
the  mailing  list.  However,  by  revealing  that  the  treatment  did  not  affect  attitudes  about 
the  mailing  list  is  favorable  in  and  of  itself,  because  it  showed  that  attitudes  about  the 
mailing  list  did  not  decrease  after  receiving  the  treatment. 
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When  testing  the  last  hypothesis  with  a  one-way  ANOVA  test,  there  were  no 
significant  differences  indicated  on  the  instructor  and  course  ratings  between  the  distance 
education  and  traditional  group.  This  is  a  positive  result,  because  it  indicates  that  the  two 
groups  had  similar  experiences  and  attitudes  towards  the  course  and  the  instructor.  This 
reveals  that  the  mode  of  educational  delivery  did  not  affect  overall  course  and  instructor 
ratings  in  this  particular  experiment. 

Limitations  of  the  Research 

Before  detailing  the  recommendations  for  further  research,  it  is  important  to  state 
some  of  the  general  research  observations  made  by  the  researcher.  This  was  the  first  time 
the  instructor  not  only  taught  the  course  but  also  taught  any  course  by  satellite/CMC. 

This  was  also  the  first  mailing  list  course  the  researcher  had  moderated.  Needless  to  say, 
lack  of  experience  with  the  medium  for  both  the  instructor  and  the  moderator  may  have 
impacted  the  results  of  the  research  experiment. 

Some  social  and  technological  problems  were  the  encountered  with  the  mailing  list. 
In  order  to  ensure  participation,  it  was  decided  to  make  contributions  mandatory  for  the 
mailing  list.  Many  students  resented  being  required  to  contribute.  Others  only  had  access 
to  the  mailing  list  at  work,  and  stated  they  barely  had  enough  time  to  do  their  jobs,  much 
less  make  adequate  contributions  to  the  mailing  list.  Some  subscribers  were  hesitant  to 
post  messages  because  they  did  not  see  any  benefit  in  sending  e-mail  to  complain  or  to 
show  ignorance  with  the  course  subject.  Some  students  viewed  the  mailing  list  as  a 
positive  tool  to  distribute  homework  assignments  and  to  ask  questions  that  went  unasked 
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during  class  either  because  of  a  failed  audio  link  or  viewing  of  the  taped-video  broadcast. 
In  general,  students  felt  that  the  mailing  list  should  be  used  to  enhance  the  course,  but  not 
be  included  as  a  graded  portion  of  the  course. 

The  moderated  mailing  list  program  itself  was  cumbersome  to  moderate  and  some  of 
the  receiving  sites  had  difficulty  sending  in  messages  for  posting.  Other  sites  had 
difficulty  receiving  messages  that  were  posted  to  the  mailing  list.  As  these  problems  were 
encountered,  the  moderator  and  systems  administrator  handled  them  to  the  best  of  their 
ability.  However,  in  dealing  with  other  base  systems  administrators,  some  problems  were 
not  solved  until  half-way  through  the  quarter.  Even  though  the  mailing  list  requirements 
were  given  out  and  briefed  by  the  instructor  on  the  first  day  of  class,  some  students  did 
not  subscribe  to  the  mailing  list  until  late  in  the  quarter.  The  number  of  postings  by  each 
individual  and  the  date  they  subscribed  to  the  mailing  list  can  be  found  at  Appendix  D.  The 
moderator  forwarded  all  previously  posted  mailing  list  messages  to  the  late  subscribers,  as 
well  as  to  the  subscribers  who  were  experiencing  technical  difficulties. 

A  couple  of  the  base  education  centers’  test  administrators  did  not  administer  the 
pre-course  survey  at  the  same  time  as  the  pretest.  The  course  administrator  realized  there 
may  have  been  some  miscommunication,  and  resent  some  pre-course  surveys  to  a  few  of 
the  remote  sites  during  the  third  week  of  class.  Some  of  the  pre-course  survey  data  may 
be  tainted  because  the  treatment  was  already  being  administered  (the  mailing  list/satellite 
course)  when  the  base  education  centers  administered  the  pre-course  survey. 

Three  of  the  nineteen  remote  sites  did  not  receive  a  live  satellite  broadcast.  The 
original  intent  of  this  research  project  was  to  compare  the  taped-video,  satellite  and 
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traditional  methods.  However,  since  there  were  only  six  students  from  the  taped  sites,  and 
only  three  of  the  six  turned  in  pre-  or  post-course  surveys,  it  was  decided  that  the  sample 
size  for  the  video  portion  was  not  sufficient.  Therefore,  the  taped-video  students  were 
considered  with  the  satellite  distance  education  group. 

Recommendations  for  Further  Research 

It  is  recommended  that  this  research  project  be  conducted  on  other  professional 
continuing  education  courses  to  determine  if  the  results  can  be  applied  to  other  non¬ 
software  engineering  courses. 

The  requirement  to  use  the  mailing  list  may  have  discouraged  students  from 
participating.  It  would  be  interesting  to  conduct  the  experiment  again  without  the 
mandatory  mailing  list  participation  requirement  --  encouraging  mailing  list  participation  in 
other,  more  positive  ways. 

By  duplicating  this  study  on  the  same  course  with  the  same  instructor,  with  an 
experienced  mailing  list  moderator,  the  results  may  be  more  indicative  of  the  treatment 
and  not  the  inexperience.  The  study  should  also  be  repeated  on  groups  with  no 
significant  demographic  differences,  to  fiarther  examine  the  grade  differences  between 
groups.  Perhaps  groups  with  similar  backgrounds  will  have  no  significant  grade 
differences  between  the  distance  education  and  traditional  groups. 
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Summary 

The  literature  dealing  with  satellite  distance  education  consistently  indicates  that  the 
delivery  of  educational  programming  via  satellite  broadcast  is  educationally  effective. 
However,  typical  satellite  distance  education  methods  have  always  had  one  big  drawback  - 
-  limited  interaction.  Some  academic  institutions,  like  the  Air  Force  Institute  of 
Technology,  are  using  computer-mediated  communication  to  provide  its  remote  students 
with  a  tool  to  interact  during  non-class  hours.  This  was  the  first  deliberate  attempt  by  the 
Air  Force  Institute  of  Technology  to  electronically  communicate  with  its  satellite  distance 
education  students. 

The  objective  of  this  research  was  to  examine  an  Air  Force  software  engineering 
Professional  Continuing  Education  distance  education  course  to  determine  if  computer- 
mediated  communication  supplemented  satellite  distance  education  delivery  is  as  good  as 
or  better  than  the  traditional  face-to-face  education  delivery  method.  In  particular,  this 
study  examined  whether  the  CMC  supplemented  satellite  distance  education  course 
performance,  interaction,  and  attitude  outcomes  were  at  least  as  good  as  the  traditional 
face-to-face  education  course  outcomes. 

This  research  effort  used  a  non-equivalent  control  group  methodology  to  analyze  the 
performance,  interaction,  and  attitude  outcomes  of  the  Air  Force  Institute  of  Technology’s 
School  of  Systems  and  Logistics  Software  Engineering  Departments’  (AFIT/LSS)  Object 
Oriented  Analysis  and  Design  (CSE  494)  Software  Professional  Development  Program 
course.  The  results  of  the  demographic  and  computer  attitudes  data  revealed  that  the  two 
groups  were  significantly  different  in  only  two  areas-  educational  level  and  self-rated 


52 


computer  proficiency  ~  with  the  traditional  group  rating  higher  on  both  accounts.  The 
significant  differences  on  these  items  can  give  insight  to  the  observed  differences  found 
between  the  two  groups  on  performance  outcomes.  Significant  differences  were  found 
between  the  groups  on  the  pretest,  final  exam  and  overall  course  grades,  but  not  on  the 
midterm  grades.  With  the  traditional  group  having  higher  education  levels  and  self- 
assessed  computer  proficiency  ratings,  it  is  not  surprising  that  they  scored  higher  on  the 
pretest  and  final  than  the  distance  education  group  did.  However,  this  assessment  does 
not  explain  the  midterm  grades.  When  examining  midterm  grades,  there  were  no 
significant  differences  between  the  two  groups,  indicating  that  distance  education  students 
can  perform  as  well  as  traditional  students  on  certain  measurements  of  academic 
performance. 

When  examining  the  interaction  outcomes,  there  were  no  significant  differences  on 
student-to-student  and  student-to-instructor  interaction  rates  between  the  two  groups. 

This  indicates  that  the  mailing  list  enabled  the  distance  education  group  to  achieve 
comparative  interaction  rates  to  the  traditional  group.  By  providing  remote  students  with 
a  communication  tool  that  allows  interaction  during  non-class  hours,  the  feelings  of 
isolation  and  solitude  for  the  remote  student  can  be  lessened  and  possibly  eliminated. 

After  examination  of  attitudes  towards  computers  and  expectations  towards  the 
mailing  list  with  participation  rates  for  the  distance  education  students,  it  was  revealed  that 
there  were  no  significant  dififerences  between  positive  and  negative  computer  attitudes  and 
expectations  for  the  mailing  list  with  mailing  list  participation  rates.  This  is  somewhat 
surprising  because  other  research  has  indicated  that  more  positive  pre-course  attitudes 
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towards  computers  and  the  educational  medium  will  lead  to  higher  participation  rates. 
However,  there  were  many  other  factors  that  may  have  affected  participation  rates  that 
this  researcher  either  did  not  study  or  could  not  study. 

Upon  conclusion  of  the  experiment,  this  researcher  examined  pre-course  and  post¬ 
course  attitudes  towards  the  mailing  list.  It  was  discovered  that  there  were  no  significant 
differences  between  the  pre-course  expectations  about  using  the  mailing  list  and  post¬ 
course  experiences  with  the  mailing  list.  In  other  words,  the  treatment  did  not  affect  the 
distance  education  students’  attitudes  towards  the  mailing  list. 

When  examining  instructor  and  course  ratings,  the  two  groups  had  similar 
experiences  and  attitudes  towards  the  course  and  the  instructor,  revealing  that  the  mode  of 
educational  delivery  does  not  affect  overall  satisfaction  with  the  course  and  the  instructor. 
This  indicates  that  the  effectiveness  of  the  instruction  was  perceived  to  be  the  same  by 
both  the  distance  education  group  and  the  traditional  group. 

Despite  some  negative  findings  in  relation  to  course  performance,  the  overall  results 
of  this  study  were  encouraging.  They  suggest  that  satellite  distance  education  that  is 
supplemented  with  computer-mediated  communication  can  be  an  effective  educational 
delivery  method  for  software  engineering  Professional  Continuing  Education  courses. 

After  examining  some  of  the  limitations  of  this  research  project  —  instructor  inexperience 
with  the  class  and  delivery  medium,  researcher  inexperience  with  the  mailing  list,  student 
dissatisfaction  with  the  mandatory  participation  requirement,  technical  problems  with  the 
mailing  list  and  late  administration  of  some  pre-course  surveys  —  further  research  is 
recommended  in  particular  areas.  Some  of  these  areas  include:  investigating  similar 
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research  experiments  that  incorporate  positive  encouragement  for  mailing  list 
participation,  duplicating  this  research  experiment  with  the  same  instructor  (who  is  now 
experienced  with  the  course  and  the  medium)  and  course,  and  conducting  a  similar 
experiment  on  two  groups  with  no  significant  demographic  differences. 

With  the  continuing  need  to  reach  more  students  with  less  money,  it  is  very 
important  for  the  Air  Force  to  develop  an  effective  distance  education  method  that  allows 
for  interaction.  Satellite  distance  education  is  now  available  at  most  Air  Force  bases,  and 
with  more  and  more  bases  becoming  connected  to  the  Internet,  the  possibilities  for 
interaction  using  computer-mediated  communication  are  endless.  Distance  education 
initiatives,  like  the  one  here  at  the  Air  Force  Institute  of  Technology,  need  to  be  evaluated 
and  improved  so  that  the  greatest  benefit  from  educational  technology  can  be  realized. 
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Appendix  A:  Software  Engineering  Course  Listing 


Air  Force  Institute  of  Technology 
Software  Engineering  Department 
Software  Professional  Development  Course  Listing 


Course  Number 

Course  Title 

CSE  492 

Software  Systems  Engineering 

CSE  493 

Software  Requirements  and  Design  Engineering 

CSE  494 

Object  Oriented  Analysis  and  Design  Engineering 

CSE  495 

Software  Creation  and  Maintenance 

CSE  496 

Software  Engineering  Practicum  (offered  in 
residence  only) 
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Appendix  B:  Mailing  List  Handout 
CSE  494  Mailing  List  Handout 

The  Object-Oriented  Analysis  and  Design  Software  Professional  Development  Course 
will  be  using  a  mailing  list  program  to  supplement  the  satellite  and  videotaped  classroom 
sessions.  The  mailing  list  program  is  intended  to  provide  asynchronous  discussion 
sessions  for  the  distance  education  students.  Homework  assignments  will  be  distributed 
over  the  mailing  list.  Students  are  encouraged  to  ask  questions,  provide  feedback  and  give 
comments  on  the  course  and  the  medium  of  course  delivery.  Active  involvement  with  the 
mailing  list  will  count  toward  your  CSE  494  class  participation  grade. 

Distance  education  students  are  required  to: 

1 .  Subscribe  to  the  mailing  list  by  the  end  of  the  first  week. 

2.  Check  the  mailing  list  for  electronic  mail  messages  at  least  twice  a  week. 

3.  Make  adequate  contributions  to  the  mailing  list  at  least  once  every  two  weeks. 

To  subscribe  to  the  mailing  list; 

Send  electronic  mail  to:  spdp-request@afit.af  mil 

Content  of  e-mail  message:  subscribe  494  “your  name” 

To  post  messages  to  the  mailing  list: 

Send  your  topic  or  questions  for  discussion  in  an  e-mail  message  to  this  address: 
494@lss.afit.afmil 

All  of  the  current  mailing  list  subscribers  will  receive  a  copy  of  your  e-mail  message. 

To  contact  Capt.  Cecil  with  individual  messages,  send  e-mail  to: 

dcecil@l  ss .  afit.  af  mil 

Note:  The  instructor  may  forward  your  electronic  mail  to  the  mailing  list  if  your  message 
will  add  value  to  the  discussion  session. 

All  subscribers  to  the  mailing  list  will  receive  the  electronic  mail  messages  that  are  sent  to 
the  mailing  list.  This  mailing  list  is  moderated.  All  messages  will  be  approved  by  the 
moderator  before  the  message  is  posted  to  the  mailing  list.  Therefore,  there  may  be  some 
lag  time  between  when  you  send  a  message  and  when  it  actually  is  posted  to  the  mailing 
list.  The  mailing  list  will  be  checked,  at  a  minimum,  each  day  by  1 100. 

The  moderator  for  this  mailing  list  is  Capt.  Megan  C.  Block.  Questions  about  the  mailing 
list  should  be  e-mailed  to:  curranm@lss.afit.af  mil. 

Distance  education  students  without  access  to  electronic  mail  need  to  contact  the 
mailing  list  moderator.  Capt.  Block  can  be  reached  at  (513)  427-2731  or  DSN  785- 
7777  (ext.  2123).  An  alternate  method  to  receive  the  information  will  be  discussed. 
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Appendix  C:  Requirements.  Timeframe  and  POC  Listing 

Software  Engineering  Distance  Education  Research  Project 


Requirement 

1.  Develop  pretest 

Timeframe 

By  30  June 

POC 

Capt  Cecil 

2.  Develop  pre-course  survey 

By  30  June 

Capt  Block 

3.  Send  out  Pretest  and  pre-course  Survey 

By  3  July 

Course  Admin/Capt  Block 

4.  Administer  Pretest  and  pre-course  Survey 

2nd  day  of  class 

Test  Admin/Capt  Cecil. 

5.  Discuss  Mailing  List  with  Students  1st  day  of  class 

*How  to  subscribe  1st  day  of  class 

♦Who  can  subscribe  (DE  only,  not  WP  students)  1st  day  of  class 
♦What  type  of  info  will  be  sent  out  by  Capt  Cecil  1st  day  of  class 

♦Students  need  to:  subscribe  to  Mailing  List  By  end  of  1st  week 

check  mailing  list  Twice  a  week 

make  adequate  contributions  Bi-weekly 

Capt  Cecil 

Capt  Cecil 

Capt  Cecil 

Capt  Cecil 

DE  Student 

DE  Student 

DE  Student 

6.  Get  info  to  DE  students  that  don’t  have  e-mail 

Day  of  dissemination 

Capt  Block 

7.  Get  info  to  WP  students  not  allowed  access 

Before/after  class 

Capt  Cecil/Capt  Block 

8.  Perform  admin  on  moderated  Mailing  List 
♦Subscribe  students 
♦Forward  appropriate  messages 
♦Work  with  Sys  admin  on  errors/problems 

Each  day  before  1 100 
Daily-as  needed 

Daily-as  needed 

Daily-as  needed 

Capt  Block 

Capt  Block 

Capt  Block 

Capt  Block 

9.  Contribute  info  for  Mailing  List 

Cecil 

2-3  Times  a  week  Capt 

10.  Assist  Capt  Cecil  with  homework 

as  needed 

Capt  Block 

11.  Grade  Pretest 

Before  midterm 

Course  Admin/Capt  Cecil 

12.  Perform  Stat  Analysis  on  pre-course  survey 

Before  midterm 

Capt  Block 

13.  Develop/send  out  midterm 

11  Aug 

Capt  Cecil/Course  Admin 

14.  Administer  midterm 

18  Aug 

Test  Administrator 

15.  Grade  Midterm 

Just  after  midterm 

Course  Admin/Capt  Cecil 

16.  Develop  post-course  survey/final 

Before  midterm 

Capt  Block/Capt  Cecil 

17.  Send  out  final  and  post-course  survey 

15  Sept 

Course  Admin/Capt  Block 

18.  Administer  final  and  post-course  survey 

18  Sept 

Test 

Administrator 
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19.  Grade  Final 


Just  after  final 


Course  Admin/Capt  Cecil 


20.  Perform  Stat  Analysis  on  post-course  survey  Just  after  final  Capt  Block 

Appendix  D:  CSE  494  Mailing  List  Postings 


Type 

SSAN 

Participation 

Date  signed  Up 

DE 

1360 

2 

25-Jul 

DE 

5671 

2 

1-Aug 

DE 

6647 

5 

10-Jul 

DE 

9792 

26-Jul 

DE 

3030 

20-Jul 

DE 

3475 

1 

11 -Ju! 

DE 

9796 

1 

19-Jul 

DE 

214 

11-Jul 

DE 

9831 

1 

12-Jul 

DE 

2481 

4 

17-Jul 

DE 

4244 

20-Jul 

DE 

1587 

12-Jul 

DE 

6270 

4 

11-Jul 

DE 

9753 

14-Jul 

DE 

9140 

2 

14-Jul 

DE 

7099 

1 

17-Jul 

DE 

3763 

11-Jul 

DE 

6564 

1 

11-Jul 

DE 

3395 

20-Jul 

DE 

3824 

13-Jul 

DE 

3110 

1 

11-Jul 

DE 

2890 

24-Jul 

DE 

4375 

3 

19-Jul 

DE 

7213 

19-Jul 

DE 

4213 

3 

10-Aug 

DE 

5331 

1 

18-Jul 

DE 

6626 

3 

25-Jul 

DE 

5080 

14-Jul 

DE 

5539 

1 

11-Jul 

DE 

9947 

1 

12-JuI 

DE 

7192 

3 

14-Jul 

DE 

1160 

1 

11-Jul 

DE 

4944 

3 

11-Jul 

DE 

7795 

14-Jul 

DE 

588 

1 

11-Jul 

DE 

1297 

12-Jul 

DE 

580 

14-Jul 

DE 

7486 

2 

26-Jul 

***  This  doesn't  include  5  or  6  messages  sent  directly  to  Capt  Cecil  and  then  combined 
and  sent  out  on  1 8  July.  This  also  does  not  include  24  responses  to  a  survey  on  software 
engineering  conducted  by  Capt  Cecil. 
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Appendix  E:  Pre-course  survey 

Pre-course  Software  Engineering  Survey 

The  Air  Force  Institute  of  Technology  is  conducting  a  study  on  the  effectiveness  of 
distance  education  in  comparison  to  traditional  education.  The  Software  Engineering  Department 
of  the  School  of  Systems  and  Logistics  has  approved  research  on  their  software  engineering 
courses.  This  survey  is  one  of  the  instruments  that  will  be  used  in  the  study  of  the  effectiveness  of 
your  software  engineering  course  delivered  in  the  distance  education  mode  (supplemented  with 
computer-mediated  communication)  compared  to  the  course  offered  in  the  traditional  face-to-face 
mode.  Survey  data  will  be  collected  at  the  beginning  and  ending  of  this  course.  All  surveys  will 
be  anonymous  and  responses  will  be  non-attributable.  Your  participation  in  filling  out  this  survey 
is  strictly  voluntary  and  greatly  appreciated.  There  will  be  no  impact  on  your  grade  if  you  choose 
not  to  participate.  Please  answer  all  survey  questions  completely  and  honestly. 

Thank  you  for  your  time  and  effort. 

Captain  Dan  Cecil 
Captain  Megan  Curran  Block 

Directions;  Indicate  your  responses  by  answering  in  the  blanks  provided  or  by  circling  the 
appropriate  response. 

Name  (optional) _ 

Last  four  digits  of  your  social  security  numljer  (for  coding  purposes  only)  _ 

I.  Male _ Female _  2.  Age _ 

3.  Are  you  military  or  civilian?  Military _  Civilian _ 4.  What  is  your  Rank  or  Grade? _ 

5.  What  is  your  Air  Force  Specialty  Code  or  job  series? _ 

6.  What  is  your  current  job  title? _ 

7.  How  long  have  you  been  assigned  to  your  current  job? _ years _ months 

8.  Is  your  current  job  related  to  software  engineering?  yes _  no _ 

9.  How  long  have  you  worked  in  the  software  engineering  field? _ years _ months _ N/A 

10.  What  is  your  highest  level  of  education?  (please  list  degree  type) 

Undergraduate  degree _ 

Some  Graduate  courses _ 

Graduate  degree _ 

Other _ 

II.  How  many  undergraduate  software  engineering  credits  have  you  completed? _ 

12.  How  many  graduate  software  engineering  credits  have  you  completed? _ 
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13.  How  many  professional  continuing  education  software  engineering  courses  have  you  taken? _ 

14.  How  much  out-of-class  time  do  you  expect  to  dedicate  to  this  course  each  week? 

1-less  than  one  hour  2-  one  to  two  hours  3-  two  to  three  hours  4-three  or  more  hours 

15.  Do  you  have  access  to  a  computer  that  can  connect  to  the  Internet? 

1-yes  2-  no 

16.  If  you  answered  yes  to  question  15,  where  is  the  computer  located? 

1- at  home  2- at  work  3- other _ 

17.  How  would  you  rate  your  proficiency  with  personal  computers? 

1 -no  experience  2-newuser  3 -moderately  proficient  4-proficient  5-very  proficient 

18.  Have  you  ever  taken  a  satellite  distance  education  course? 

1-  yes  2-  no 

If  yes,  From  which  institution  did  you  take  this  course?  _ _ 

19.  Have  you  ever  taken  a  satellite  course  supplemented  with  computer-mediated  communication? 

1-yes  2-no 

If  yes,  From  which  institution  did  you  take  this  course?  _ _ 

20.  How  often  have  you  used  an  electronic-mail  system? 

Frequently  Never 

1  2  3  4  5  6  7 

21.  Have  you  ever  been  a  member  of  an  electronic  mailing  list  system? 

1-yes  2- no 

For  each  of  the  following  pairs  of  words,  please  chose  the  response  that  is  the  closest  to  your  CURRENT 
FEELINGS  ABOUT  USING  COMPUTERS.  For  instance,  if  you  feel  computer  systems  in  general  are  completely 
"stimulating"  to  use  and  not  at  all  "dull,"  circle  1;  4  means  you  are  undecided  or  neutral,  or  think  they  are  equally 
likely  to  be  stimulating  or  dull;  3  means  you  feel  that  they  are  slightly  more  stimulating  than  dull,  etc. 


22. 

Stimulating 

1 

2 

3 

4 

5 

6 

7 

Dull 

23. 

Fun 

1 

2 

3 

4 

5 

6 

7 

Dreary 

24. 

Easy 

1 

2 

3 

4 

5 

6 

7 

Difficult 

25. 

Personal 

1 

2 

3 

4 

5 

6 

7 

Impersonal 

26. 

Helpful 

1 

2 

3 

4 

5 

6 

7 

Hindering 

27. 

Non  threatening 

1 

2 

3 

4 

5 

6 

7 

Threatening 

28. 

Efficient 

1 

2 

3 

4 

5 

6 

7 

Inefficient 

29. 

Obliging 

1 

2 

3 

4 

5 

6 

7 

Demanding 

30. 

Desirable 

1 

2 

3 

4 

5 

6 

7 

Undesirable 
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**If  you  are  a  Wright-Patterson  student,  please  stop  here.  Questions  3 1-39  apply  to  satellite  students  only.** 

An  electronic  mailing  list  program  allows  messages  to  be  shared  among  an  established  group  of  people. 
Individuals  who  subscribe  to  the  mailing  list  will  receive  all  electronic  mail  messages  that  are  sent  to  the  mailing 
list.  Replies  to  those  mailing  list  messages  are  also  sent  to  all  of  the  mailing  list  members.  Mailing  list  programs 
enable  sharing  information  with  a  group  a  people  who  have  related  interests. 


Indicate  your  expectations  about  using  a  mailing  list  system  by  choosing  the  number  that  best  indicates  where  your 
feelings  lie  on  the  scales  below. 

31.  Easy  to  learn  1  2  3  4  5  6  7  Hard  to  learn 

32.  Friendly  1  2  3  4  5  6  7  Impersonal 

33.  Not  frustrating  1  2  3  4  5  6  7  Frustrating 

34.  Productive  1  2  3  4  5  6  7  Not  productive 


35.  Do  you  expect  that  the  use  of  the  mailing  list  system  will  increase  the  efficiency  of  your  education  (the  quantity 
of  your  work  that  you  can  complete  in  a  given  time)? 

Definitely  yes  1  2  3  4  5  6  7  Definitely  not 


36.  Do  you  expect  the  use  of  the  mailing  list  system  will  increase  the  quality  of  your  education? 

Definitely  yes  1  2  3  4  5  6  7  Definitely  not 

37.  Do  you  resent  being  required  to  use  the  mailing  list  system  for  this  course? 

Definitely  yes  1  2  3  4  5  6  7  Definitely  not 

38.  Overall,  how  useful  do  you  expect  the  mailing  list  system  to  be  for  this  course? 

Very  Usefiil  1  2  3  4  5  6  7  Not  useful  at  all 

39.  While  you  are  part  of  this  course,  how  much  time  in  the  average  week  do  you  foresee  yourself  using  the 
mailing  list  system  in  relation  to  your  coursework? 

1-  less  than  30  minutes  2-  30  minutes  to  one  hour  3- 1  to  2  hours  4-  3  to  4  hours  5-  5  hours  or 

more 


Questions  22-39  of  this  survey  instrument  were  developed  by  Starr  Roxanne  Hiltz  for  the 
evaluation  of  the  Virtual  Classroom  (registered  trademark)  found  in  Chapter  7  (p.l34)  of  Online 
Education:  Perspectives  on  a  New  Environment,  edited  by  Linda  M.  Harasim,  1991 .  These 
questions  were  used  vsdth  permission. 

Thank  you  for  completing  this  survey.  We  appreciate  your  time  and  effort.  An  executive 
summary  of  the  study  results  can  be  provided  upon  request. 
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Appendix  F:  Pre-course  Survey  Explanation 


Explanation  of  Choices  for  Pre-course  Survey 
Questions  1-13  and  16 

1.  Sex  Male-1  Female -2 

2.  Age  XX  years 

3.  Military  -  1  Civilian  -  2 

4.  Rank  or  Grade  MSgt,  2Lt,  ILt,  GS-1 1-1 

Capt ,  Major,  GS-12,  GS-13  -  2 

5.  Civilian  Job  Series  or  Air  Force  Specialty  Code* 

Computer  Specialist  334  -  1 

Mechanical  Engineering  830  -2 
(Unknown  -  no  entry  in  the  AFP)  854  -  3 
Electronics  Engineering  855  -  4 
Mathematics  1520  -  5 
Computer  Science  1550  -  6 

Military  Communication/Computers  Officer  3351/3  -  7 
Military  Communication/Computers  NCO  3C0X1  -  8 
Other  -  9 

6.  Current  Job  Title  Mathematician  -  1 

Electronic  Engineer  -  2 

Software  Engineer  -  3 

Software  Analyst  -  4 

Computer  Engineer  -  5 

Computer  Programmer  -  6 

Computer  Scientist  -  7 

Database  Administrator  -  8 

Computer  Systems  Analyst  -  9 

Project  Manager  -  10 

Other  -  Computer  related  title  -  1 1 

Other  -  Non-computer  related  title  -  12 

7.  Length  of  time  assigned  to  current  job  XXX  months 

8.  Current  job  related  to  Software  Engineering  yes  -  1  no  -  2 

9.  Length  of  time  in  Software  Engineering  field  XXX  months 
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10.  Highest  level  of  education  Undergraduate  degree  -  1 

Some  Graduate  courses  -  2 
Graduate  degree  -  3 
Other  -  4 

1 1 .  Number  of  Undergraduate  Software  Engineering  credits  X 

12.  Number  of  Graduate  Software  Engineering  credits  X 

13.  Number  of  PCE  Software  Engineering  courses  X 

16.  Computer  location  At  home  -  1 

At  work  -  2 
Other  -  3 

At  home  and  at  work  -  4 
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Appendix  G:  Post-course  Survey 
Post-course  Software  Engineering  Survey 

The  Software  Engineering  Department  at  the  Air  Force  Institute  of  Technology  has  been 
conducting  a  study  on  the  effectiveness  of  distance  education  in  comparison  to  traditional 
education.  This  survey  is  the  last  instrument  that  will  be  used  in  the  study  of  the  effectiveness  of 
your  software  engineering  course  delivered  in  the  distance  education  mode  (supplemented  with 
computer-mediated  communication)  compared  to  the  course  offered  in  the  traditional  face-to-face 
mode.  As  mentioned  in  the  previous  study,  all  surveys  will  remain  anonymous  and  your  responses 
are  non-attributable.  Your  participation  in  this  survey  remains  voluntary  and  is  greatly 
appreciated.  Please  answer  all  survey  questions  completely  and  honestly. 

Thank  you  again  for  your  time  and  effort. 

Captain  Dan  Cecil 
Captain  Megan  Curran  Block 

Directions;  Indicate  your  responses  by  answering  in  the  blanks  provided  or  by  circling  the 
appropriate  response. 

Name  (optional) _ Location _ 

Last  four  digits  of  your  social  security  number  (for  coding  purposes  only)  _ 

Primary  Method  of  Delivery:  Satellite _ Live  Lecture _ Taped  Lecture _ 

1.  How  much  out-of-class  time  did  you  dedicate  to  this  course  each  week? 

1-less  than  one  hour  2-  one  to  two  hours  3-  two  to  three  hours  4-three  or  more  hours 

2.  How  often  did  you  communicate  with  other  students  outside  of  class,  by  computer,  “face-to-face”  or  on  the 
telephone? 

1-never  2-rarely  3-sometimes  4-often  5-firequently  6-constantly 

3.  How  often  did  you  communicate  with  your  instructor  outside  of  class,  by  computer,  “fece-to-face”  or  on  the 
telephone? 

1-never  2-rarely  3-sometimes  4-often  5-frequently  6-constantly 

For  the  next  three  questions,  choose  the  number  that  best  describes  where  your  feelings  lie  on  the  scale  listed 


below  each  statement. 

4.  Students  in  my  class  tended  to  be: 

Extremely  cooperative  12  3 

4 

5 

6 

Not  at  all  cooperative 

5.  The  help  I  received  from  other  students  was: 
Crucially  important  to  me  1  2  3 

4 

5 

6 

Useless  or  misleading 

6.  This  course  was  more  of  a/an: 

Individual  experience  12  3 

4 

5 

6 

Group  experience 

7.  The  instructor  organized  this  course  well. 

1 -Strongly  agree  2- Agree 

3-Not  sure 

4-Disagree  5-Strongly  disagree 
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8.  Grading  was  fair  and  impartial. 


1 -Strongly  agree  2- Agree  3 -Not  sure 

4-Disagree 

5-Strongly  disagree 

9.  The  instructor  seems  to  enjoy  teaching. 

1-Strongly  agree  2-Agree  3-Not  sure 

4-Disagree 

5-Strongly  disagree 

10.  The  instructor  has  sufficient  knowledge  about  this  subject  area. 

1  -Strongly  agree  2-Agree  3  -Not  sure 

4-Disagree 

5-Strongly  disagree 

11.  Students  were  encouraged  to  express  ideas. 

1-Strongly  agree  2-Agree  3-Not  sure 

4-Disagree 

5-Strongly  disagree 

12.  The  instructor  presented  material  clearly  and  summarized  main  points. 

1-Strongly  agree  2-Agree  3-Not  sure  4-Disagree 

5-Strongly  disagree 

13.  The  instructor  discussed  points  of  view  other  than  his  own. 
1-Strongly  agree  2-Agree  3-Not  sure 

4-Disagree 

5-Strongly  disagree 

14.  Students  were  able  to  get  personal  help  from  the  instructor  in  this  course. 

1-Strongly  agree  2-Agree  3-Not  sure  4-Disagree 

5-Strongly  disagree 

15.  The  instructor  presented  the  material  in  an  interesting  manner. 
1-Strongly  agree  2-Agree  3-Not  sure 

4-Disagree 

5-Strongly  disagree 

16.  The  instructor  critiqued  my  work  in  a  constructive  and  helpful  way. 

1-Strongly  agree  2-Agree  3-Not  sure  4-Disagree 

5-Strongly  disagree 

17.  Overall,  I  would  rate  this  instructor  as: 

1-Excellent  2-Very  good  3-Good 

4-Fair 

5-Poor 

18.  This  course  made  me  more  interested  in  the  subject. 

1-Strongly  agree  2-Agree  3-Not  sure 

4-Disagree 

5-Strongly  disagree 

19.  I  learned  a  great  deal  of  factual  and  useful  material. 

1-Strongly  agree  2-Agree  3-Not  sure 

4-Disagree 

5-Strongly  disagree 

20.  I  gained  a  good  understanding  of  basic  concepts. 

1-Strongly  agree  2-Agree  3-Not  sure 

4-Disagree 

5-Strongly  disagree 

21.  I  learned  to  identify  central  issues  in  this  field. 

1-Strongly  agree  2-Agree  3-Not  sure 

4-Disagree 

5-Strongly  disagree 

22.  I  developed  the  ability  to  communicate  clearly  about  this  subject. 
1-Strongly  agree  2-Agree  3-Not  sure 

4-Disagree 

5-Strongly  disagree 

23.  This  course  was  a  good  use  of  my  time. 

1-Strongly  agree  2-Agree  3-Not  sure 

4-Disagree 

5-Strongly  disagree 

24.  I  developed  new  friendships  in  this  class. 

1-Strongly  agree  2-Agree  3-Not  sure 

4-Disagree 

5-Strongly  disagree 
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25.  I  learned  to  value  other  points  of  view. 

1-Strongly  agree  2-Agree  3-Not  sure 

4-Disagree 

5-Strongly  disagree 

26.  Overall,  I  would  rate  this  course  as: 

1 -Excellent  2-Veiy  good  3-Good 

4-Fair 

5-Poor 

**If  you  are  a  Wright-Patterson  student,  please  stop  here.  Questions  27-37  apply  to  satellite  students  only.** 

27.  How  much  time  in  the  average  week  did  you  use  the  mailing  list  system  in  relation  to  your  coursework? 

1-  less  than  30  minutes  2-  30  minutes  to  one  hour  3- 1  to  2  hours  4-  3  to  4  hours  5-  5  hours  or 

more 


Indicate  your  experiences  with  the  mailing  list  system  by  choosing  the  number  that  best  describes  where  your 
feelings  lie  on  the  scales  below. 


28.  Easy  to  learn  1 

2 

3 

4 

5 

6 

7 

Hard  to  learn 

29.  Friendly  1 

2 

3 

4 

5 

6 

7 

Impersonal 

30.  Not  frustrating  1 

2 

3 

4 

5 

6 

7 

Frustrating 

31.  Productive  1 

2 

3 

4 

5 

6 

7 

Not  productive 

32.  Did  the  use  of  the  mailing  list  system 

increase  the  efficiency  of  your  education  (the  quantity  of  yoin  work  that 

you  completed  in  a  given  time)? 

Definitely  yes  1 

2 

3 

4 

5 

6 

7 

Definitely  not 

33.  Did  the  use  of  the  mailing  list  system  increase  the  quality  of  your  education? 

Definitely  yes  1 

2 

3 

4 

5 

6 

7 

Definitely  not 

34.  Did  you  find  the  course  to  be  a  better  learning  experience  than  normal 

face-to-face  courses? 

Definitely  yes  1 

2 

3 

4 

5 

6 

7 

Definitely  not 

35.  Did  you  learn  a  great  deal  more  because  of  the  mailing  list? 

Definitely  yes  1 

2 

3 

4 

5 

6 

7 

Definitely  not 

36.  Would  you  choose  to  take  another  course  that  is  supplemented  with  a  mailing  list? 

Definitely  yes  1  2  3  4  5  6  7  Definitely  not 

37.  Overall,  how  useful  was  the  mailing  list  system  for  this  course? 

Very  Usefiil  1  2  3  4  5  6  7  Not  usefiil  at  all 

Some  questions  of  this  survey  instrument  were  developed  by  Starr  Roxanne  Hiltz  for  the 
evaluation  of  the  Virtual  Classroom  (registered  trademark)  found  in  Chapter  7  (p.  134)  of  Online 
Education:  Perspectives  on  a  New  Environment,  edited  by  Linda  M,  Harasim,  1991.  These 
questions  were  used  with  permission. 

Thank  you  for  completing  this  survey.  We  appreciate  your  time  and  effort.  An  executive 
summary  of  the  study  results  will  be  provided  to  you  upon  request. 
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Appendix  H:  Variables  and  Measurement 
Software  Engineering  Distance  Education  Research  Project 
Variables  and  Measurement 


Dependent  Variables 
Performance  Outcomes 
Attitude  Outcomes 

Attitudes  toward  computers 
Time  expectation  about  the  course 
Time  spent  on  course 
Expectations  about  the  mailing  list/CMC 
Experience  with  the  mailing  list 
Overall  Rating  of  CMC  portion  of  course 
Time  expectation  about  the  mailing  list 
Time  spent  using  mailing  list 
Course  Rating 
Instructor  Rating 
Individual  vs.  Group  Learning 
Interaction  Outcomes 

Student-to-student 

Student-to-instructor 


Measurement 

Grades  on  Pretest,  Midterm,  Final,  Overall 

Results  of  Precourse  &  Postcourse  survey 

Pre  22-30 

Prel4 

Post  1 

Pre  31-38 

Post  28-31 

Post  32-37 

Pre  39 

Post  27 

Post  18-26 

Post  7-17 

Post  4-6 

Post  2 
Post  3 


Independent  Variables 

Characteristics  of  Individuals 

Computer  proficiency 

Pre  17 

Level  of  education 

Pre  10 

Experience  with  software  engineering  courses 

Pre  11,12,13 

Experience  with  satellite  courses 

Pre  18 

Experience  with  satellite/CMC  courses 

Pre  19 

Experience  with  e-mail,  mailing  list 

Pre  20,21 

Age 

Pre  2 

Military  or  Civilian 

Pre  3 

Rank/Grade 

Pre  4 

Sex 

Pre  1 

AFSC  or  job  series 

Pre  5 

Current  job  title 

Pre  6 

Time  in  current  job 

Pre  7 

Current  job  related  to  SAV  Engr. 

Pre  8 

Time  working  in  SAV  Engr.  field 

Pre  9 

Context  variables 

Primary  mode  of  course  delivery 

Traditional  face-to-face 

Post  -intro 

Satellite/CMC 

Post  -intro 

Number  of  students  at  each  site 

From  Denise 

Instructor’s  experience  with  medium 

As  determined  by 

Instructor’s  experience  with  course 

Convenient  access  to  a  computer  with  Internet 

As  determined  by 

connection 

Pre  15,16 

Intervening  variables 

Technical  problems  with  the  satellite  broadcast 
Technical  problems  with  the  mailing  list 

instructor 

instructor 
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